A nonlinear neural framework, called the generalized Hopfield network (GHN), is proposed, which is able to solve in a parallel distributed manner systems of nonlinear equations. The method is applied to the general nonlinear optimization problem. We demonstrate GHNs implementing the three most important optimization algorithms, namely the augmented Lagrangian, generalized reduced gradient, and successive quadratic programming methods. The study results in a dynamic view of the optimization problem and offers a straightforward model for the parallelization of the optimization computations, thus significantly extending the practical limits of problems that can be formulated as an optimization problem and that can gain from the introduction of nonlinearities in their structure (e.g., pattern recognition, supervised learning, and design of content-addressable memories).