Timothy P. Lillicrap is a Canadian neuroscientist and artificial intelligence researcher. He is a staff research scientist at Google DeepMind and an adjunct professor at University College London. His research centers on machine learning and statistics for optimal control and decision making, and he uses these mathematical frameworks to investigate how the brain learns. He has developed algorithms for applying deep neural networks to reinforcement learning and has introduced recurrent memory architectures for one-shot learning.
Lillicrap has been involved in the AlphaGo and AlphaZero projects at DeepMind, which achieved mastery in the games of Go, Chess, and Shogi. His contributions have earned him several honors, including the Governor General's Academic Medal, an NSERC Fellowship, the Centre for Neuroscience Studies Award for Excellence, and multiple European Research Council grants.
Early Life and Education
Lillicrap earned a B.Sc. in cognitive science and artificial intelligence from the University of Toronto in 2005. He then pursued a Ph.D. in systems neuroscience at Queen's University, completing it in 2012 under the supervision of Stephen H. Scott. His doctoral thesis, titled "Modelling Motor Cortex using Neural Network Controls Laws," was submitted to the Centre for Neuroscience Studies at Queen's University.
Career at DeepMind
After his Ph.D., Lillicrap worked as a postdoctoral research fellow at Oxford University. In 2014, he joined Google DeepMind as a research scientist. He was promoted to staff research scientist in 2016, a position he continued to hold as of 2021. In 2016, he also accepted an adjunct professorship at University College London.
At DeepMind, Lillicrap contributed to the development of the deep deterministic policy gradient (DDPG) algorithm, a method for continuous control with reinforcement learning. He co-authored the 2015 paper "Continuous Control with Deep Reinforcement Learning," which introduced DDPG and demonstrated its effectiveness on simulated robotic tasks. This work has been influential in the field of artificial intelligence and robotics.
Research Contributions
Lillicrap's research spans several areas of machine learning and neuroscience. He has worked on memory-based control with recurrent neural networks, exploring how these architectures can support learning and decision making. His 2015 paper with Nicolas Heess and David Silver addressed memory-based control, and later work with Jack W. Rae and Peter Dayan introduced fast parametric learning with activation memorization.
He was also a co-author on the 2016 paper "Asynchronous Methods for Deep Reinforcement Learning," which introduced the A3C algorithm, and on several papers on continuous deep Q-learning and robotic manipulation. His work on meta-learning, such as "Learning to Learn without Gradient Descent by Gradient Descent," has contributed to understanding how neural networks can acquire new skills efficiently.
AlphaGo and AlphaZero
Lillicrap was a co-author on the landmark 2016 Nature paper "Mastering the game of Go with deep neural networks and tree search," which described the AlphaGo system. He also contributed to the 2017 Nature paper "Mastering the game of Go without human knowledge," which introduced a version of AlphaGo that learned solely through self-play. In 2017 and 2018, he was part of the team that developed AlphaZero, a general reinforcement learning algorithm that mastered chess, shogi, and Go through self-play, as described in the Science paper "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play."
Awards and Recognition
Lillicrap has received numerous awards throughout his career. These include the NSERC Fellowship, the Governor General's Academic Medal, and the Centre for Neuroscience Studies Award for Excellence. He has also won the Social Learning Strategies Tournament twice and received a European Research Council Proof of Concept Grant. His early academic achievements were recognized with the University College Howard Ferguson Entrance Scholarship and the HPCVL / Sun Microsystems of Canada, Inc. Scholarship in Computational Sciences and Engineering.
Selected Publications
Lillicrap has an extensive publication record. Key works include:
- Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2015). "Continuous Control with Deep Reinforcement Learning." arXiv:1509.02971.
- David Silver, Aja Huang, Chris J. Maddison, et al. (2016). "Mastering the game of Go with deep neural networks and tree search." Nature, Vol. 529.
- Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, et al. (2016). "Asynchronous Methods for Deep Reinforcement Learning." arXiv:1602.01783.
- David Silver, Julian Schrittwieser, Karen Simonyan, et al. (2017). "Mastering the game of Go without human knowledge." Nature, Vol. 550.
- David Silver, Thomas Hubert, Julian Schrittwieser, et al. (2018). "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play." Science, Vol. 362, No. 6419.
References
Content in this article was adapted from Timothy Lillicrap at the Chess Programming wiki, which is licensed under the Creative Commons Attribution-Share Alike 3.0 (Unported) (CC-BY-SA 3.0) license.