Jorge Nocedal (born 1952) is an applied mathematician and computer scientist who serves as the Walter P. Murphy Professor at Northwestern University. He is recognized for his contributions to nonlinear optimization, particularly the L-BFGS algorithm and the textbook Numerical Optimization, which have become foundational in both deterministic and stochastic settings. His work has directly influenced modern Machine learning and Deep learning applications, including image and speech recognition, recommendation systems, and search engines.
Nocedal's research bridges theoretical optimization and practical algorithmic designched optimization problems that underpin training of Neural network models. His contributions are widely cited in the development of optimization software and are essential to the efficiency of large-scale learning systems used by modern Large language model platforms.
Biography
Nocedal was born and raised in Mexico. He earned a B.Sc. in physics from the National University of Mexico in 1974, followed by a PhD in mathematical sciences from Rice University in 1978, where he was supervised by Richard A. Tapia. From 1978 to 1981, Nocedal served as an assistant professor at the National University of Mexico. He then spent two years (1981–1983) as a research assistant at the Courant Institute of Mathematical Sciences at NYU.
In 1983, Nocedal joined the Electrical Engineering and Computer Sciences department at Northwestern University, where he remained until 2012. He subsequently moved to the Industrial Engineering and Management Sciences department, serving as the David and Karen Sachs Professor and chair from 2013 to 2017. He has held the Walter P. Murphy professorship, reflecting his ongoing influence in computational science.
Research Contributions
Nocedal is best known for his work on limited-memory BFGS (L-BFGS), a quasi-Newton method widely used for large-scale optimization problems. L-BFGS has become a standard tool in Artificial intelligence research, particularly for training models where full Hessian computations are infeasible. His textbook Numerical Optimization, co-authored with Stephen J. Wright, is a standard reference in the field.
His research spans both deterministic and stochastic optimization, addressing challenges in Data Augmentation and Gradient Clipping settings that are common in modern learning systems. He has explored equilibrium problems with applications in robotics, traffic flow, and games, as well as optimization in finance and PDE-constrained problems. These contributions have informed practical algorithms used in Stochastic Gradient Descent Variants and Adam (Optimizer) development, influencing how Neural network training is conducted.
In 2001, Nocedal co-founded Ziena Optimization Inc. and co-developed the KNITRO software package, a leading tool for nonlinear optimization. He served as chief scientist at Ziena from 2002 to 2012, before the company was acquired by Artelys in 2015. KNITRO is widely used in industry and academia for solving complex optimization problems.
Awards and Honors
Nocedal has received numerous awards throughout his career. In 1998, he was an invited speaker at the International Congress of Mathematicians in Berlin. He was named an ISI Highly Cited Researcher in 2004 and a SIAM Fellow in 2010. He received the Charles Broyden Prize in 2009 and the George B. Dantzig Prize in 2012. In 2017, Nocedal was awarded the INFORMS John Von Neumann Theory Prize, recognizing his fundamental contributions to optimization theory.
In 2020, he was elected a member of the National Academy of Engineering for his contributions to the theory, design, and implementation of optimization algorithms and machine learning software. This honor highlights the practical impact of his work on fields ranging from operations research to Generative AI technologies.
Legacy and Impact
Nocedal's algorithms are embedded in widely used optimization libraries and commercial software. His work on L-BFGS, a limited-memory quasi-Newton method, has become a standard tool for large-scale problems in scientific computing and statistics. The KNITRO software package, co-developed with Ziena Optimization Inc., is used across industries for solving nonlinear programming problems, including applications in robotics, finance, and PDE-constrained optimization.
As of the 2020s, his methods are also relevant to the training of Neural network models, where second-order information can improve convergence over basic stochastic gradient approaches. Although many modern Deep learning frameworks rely on adaptive methods like Adam (Optimizer), L-BFGS remains important for specific problem classes, such as those with smooth objectives and small batch sizes.
Selected Honors
Nocedal's honors include being an invited speaker at the 1998 International Congress of Mathematicians in Berlinholf and being named an ISI Highly Cited Researcher in 2004. He received the Charles Broyden Prize in 2009)Skip and the George B. Dantzig Prize in 2012. He was made a SIAM Fellow in 2010ton, received the INFORMS John Von Neumann Theory Prize in 2017, and was elected to the National Academy of Engineering in 2020 for contributions to the theory, design, and implementation of optimization algorithms and machine learning software.
Software and Industry Impact
In 2001, Nocedal co-founded Ziena Optimization Inc., where he served as chief scientist from 2002 to 2012. The company developed KNITRO, a software package for nonlinear optimization widely used in engineering and operations research. Ziena was acquired by Artelys in 2015, and KNITRO continues to be maintained under that ownership. These efforts have linked academic research with practical tools used across Amazon Web Services and other cloud platforms for optimization tasks.
Selected Publications
Nocedal's publication record includes over a hundred papers and the influential textbook Numerical Optimization (1999, second edition 2006 with Stephen J. Wright). His papers on L-BFGS and quasi-Newton methods have been cited tens of thousands of times, shaping the field of optimization. He has also published on large-scale algorithm implementations, stochastic optimization, and software design for solvers.
He has mentored numerous students and researchers who have gone on to contribute to industrial research labs, including organizations like Google Cloud, Amazon Web Services, and OpenAI the broader Artificial intelligence ecosystem.
Legacy and Impact
The algorithms developed by Nocedal are embedded in many optimization libraries used across Machine learning frameworks. The L-BFGS method, in particular, remains a reliable choice for problems with limited memory, making it relevant in settings such as Model Pruning and Curriculum Learning where resource efficiency is critical. His textbook remains a standard reference for graduate students and practitioners in operations research and computer science.
Nocedal's election to the National Academy of Engineering reflects his dual contributions to theory and software implementation. His work has enabled advances in applications ranging from Tesla to Google DeepMind research, where robust optimization is essential for Reinforcement learning and other advanced paradigms. His ongoing research continues to address the stochastic optimization challenges inherent in Large language model training.