Andrew Gehret Barto (born 1948) is an American computer scientist and professor emeritus of computer science at the University of Massachusetts Amherst. He is widely recognized as a pioneer of modern computational reinforcement learning, a field that underpins many advances in artificial intelligence and machine learning. His work, particularly with former doctoral student Richard S. Sutton, established the conceptual and algorithmic foundations for how agents learn from rewards in uncertain environments.
Barto's research has been instrumental in moving reinforcement learning from theoretical ideas to practical algorithms, influencing areas from robotics to game playing. His textbook, Reinforcement Learning: An Introduction, has become a standard reference in the field, and his contributions were recognized with the 2024 ACM A.M. Turing Award, often called the 'Nobel Prize of computing'.
Early Life and Education
Andrew Gehret Barto was born in 1948. He initially studied naval architecture and engineering at the University of Michigan, but after encountering the work of Michael Arbib, Warren Sturgis McCulloch, and Walter Pitts, he became fascinated with using computers and mathematics to model the brain. He switched his focus and earned a B.S. with distinction in mathematics in 1970. He continued at Michigan, completing a Ph.D. in computer science in 1975 with a thesis on cellular automata.
Academic Career at UMass Amherst
In 1977, Barto joined the College of Information and Computer Sciences at the University of Massachusetts Amherst as a postdoctoral research associate. He rose through the ranks, becoming an associate professor in 1982 and a full professor in 1991. He served as department chair from 2007 to 2011 and was a core faculty member of the Neuroscience and Behavior program.
At UMass, Barto co-directed the Autonomous Learning Laboratory (initially the Adaptive Network Laboratory), which became a hub for reinforcement learning research. His doctoral student, Richard Sutton, collaborated with him on key ideas, and together they developed the mathematical framework that would define the field.
Foundations of Reinforcement Learning
When Barto began at UMass, he joined a group exploring how the behavior of neurons in the brain could serve as a basis for intelligence, a concept advanced by A. Harry Klopf. Barto and Sutton used Markov decision processes (MDPs) as a mathematical foundation to explain how agents make decisions in stochastic environments, receiving rewards after each action. Traditional MDP theory assumed agents had complete knowledge of the environment; Barto and Sutton's innovation allowed for unknown environments and rewards, making the algorithms applicable to a wide range of real-world problems.
Their work laid the groundwork for modern reinforcement learning, which later powered breakthroughs such as Google's AlphaGo defeating human champions. The technique has become a cornerstone of the modern AI boom, influencing deep learning and neural network research.
Publications and Influence
Barto has published over one hundred papers and chapters in journals, books, and conference proceedings. His most influential work is the textbook Reinforcement Learning: An Introduction, co-authored with Richard Sutton, first published by MIT Press in 1998, with a second edition in 2018. He also co-edited the Handbook of Learning and Approximate Dynamic Programming (Wiley-IEEE Press, 2004) with Jennie Si, Warren Powell, and Don Wunch II.
His research has directly influenced many areas, including robotics, game playing, and autonomous systems, and his ideas are integral to the development of large language models and generative AI.
Awards and Honors
Barto has received numerous accolades for his contributions. He is a Fellow of the American Association for the Advancement of Science, a Fellow and Senior Member of the IEEE, and a member of the American Association for Artificial Intelligence and the Society for Neuroscience. He received the IEEE Neural Network Society Pioneer Award in 2004, the IJCAI Award for Research Excellence in 2017, and the UMass Neurosciences Lifetime Achievement Award in 2019.
In 2024, he was awarded the Turing Award from the Association for Computing Machinery jointly with Richard S. Sutton, with the citation: 'For developing the conceptual and algorithmic foundations of reinforcement learning.' This honor cemented his status as a foundational figure in the field.