Aakanksha Chowdhery is a computer scientist recognized for her work in Machine learning systems and wireless communications. She is best known as the lead author of the 2022 paper introducing the Pathways Language Model (PaLM), a Large language model developed at Google DeepMind. Her research spans efficient Deep learning architectures, distributed training, and networked systems, with applications in both cloud and edge computing.
Chowdhery's career has bridged academia and industry. She completed her PhD at Stanford University, where her dissertation focused on network optimization and wireless systems. She later joined Google Research, contributing to projects that improved the scalability and efficiency of Neural network training. Her work on PaLM, which had 540 billion parameters, demonstrated significant advances in few-shot learning and reasoning tasks, influencing subsequent developments in Generative AI.
Early Life and Education
Chowdhery was born in India and pursued her undergraduate studies in electrical engineering at the Indian Institute of Technology (IIT) Delhi, graduating in 2007. She then moved to the United States for graduate studies, earning a master's degree and a PhD in electrical engineering from Stanford University in 2012 and 2015, respectively. At Stanford, she worked under the supervision of professors in the information systems laboratory, focusing on cross-layer optimization for wireless networks and distributed algorithms.
Her doctoral research addressed challenges in resource allocation and interference management in heterogeneous networks. This work laid a foundation for her later interest in applying optimization techniques to Machine learning workloads, particularly in distributed and resource-constrained environments.
Career at Google and PaLM
Chowdhery joined Google Research in 2015 as a research scientist. Her early work at Google involved improving the efficiency of Deep learning models for on-device applications, including speech recognition and image classification. She contributed to the development of techniques for model compression and quantization, which are critical for deploying Neural networks on mobile and embedded devices.
In 2021, she became the lead author of the PaLM project, a large-scale effort to train a Transformer (architecture)-based language model with 540 billion parameters. The PaLM paper, published in April 2022, reported state-of-the-art results on numerous benchmarks, including reasoning, code generation, and translation. The model's ability to perform few-shot learning with chain-of-thought prompting was a notable highlight. Chowdhery's role involved coordinating a team of researchers and engineers, designing the training infrastructure, and addressing challenges related to Model Pruning and Gradient Clipping to ensure stable training.
The PaLM model was later integrated into Google's product offerings, and its architecture influenced subsequent models like PaLM 2 and Gemini. Chowdhery's contributions were widely cited, establishing her as a prominent figure in the Large language model research community.
Research Contributions
Beyond PaLM, Chowdhery has published extensively on topics such as Multi-Head Attention, Positional Encoding, and efficient training algorithms. She has explored the use of Curriculum Learning and Learning Rate Scheduling to improve convergence in large-scale models. Her work often emphasizes practical deployment, bridging the gap between theoretical Machine learning and real-world systems.
She has also investigated the intersection of wireless networking and AI, proposing methods for distributed inference at the edge. This includes research on Data Augmentation and Batch Normalization in federated learning settings, where data remains on local devices. Her interdisciplinary approach has been recognized with several best paper awards at conferences such as IEEE INFOCOM and ACM MobiCom.
Awards and Recognition
Chowdhery has received numerous honors, including the Google Research Award and the Stanford Graduate Fellowship. She was named a Rising Star in Electrical Engineering and Computer Science by MIT in 2016. Her work on PaLM was featured in major technology publications, and she has been an invited speaker at conferences like NeurIPS and ICML.
Later Career and Impact
As of 2024, Chowdhery continues to work in the field, though she has moved beyond Google to pursue independent research and advisory roles. She has been involved in initiatives to democratize access to Artificial intelligence tools, particularly for educational and healthcare applications. Her insights on scaling Transformer (architecture) models have informed industry practices at companies like OpenAI and Anthropic, which have built upon the architectural innovations introduced in PaLM.
Chowdhery's legacy lies in her ability to combine rigorous theoretical analysis with large-scale engineering. Her work has helped shape the modern landscape of Generative AI, making large models more accessible and efficient. She remains an influential voice in discussions about responsible AI development and the future of Deep learning.
Selected Publications
- Chowdhery, A., et al. (2022). "PaLM: Scaling Language Modeling with Pathways." arXiv preprint arXiv:2204.02311.
- Chowdhery, A., et al. (2019). "Efficient On-Device Training with Gradient Filtering." Proceedings of the 36th International Conference on Machine Learning (ICML).
- Chowdhery, A., & others. (2016). "Cross-Layer Optimization for Wireless Networks." IEEE Transactions on Information Theory.
These publications highlight her dual expertise in systems and algorithms, a combination that remains rare and valuable in the field of Artificial intelligence.