Amir Gholami is a computer scientist and researcher in the field of Artificial intelligence, focusing on making Deep learning models more efficient in terms of computation, memory, and energy consumption. His work addresses the growing gap between the increasing size of Neural network models and the limited hardware resources available for training and inference. Gholami is particularly known for contributions to Model Pruning, quantization, and distributed training techniques that enable large-scale AI systems to run on commodity hardware and edge devices.
Gholami's research has been published in top-tier conferences and journals, including NeurIPS, ICML, and CVPR. He has also been an active open-source contributor, releasing tools and libraries that have been widely adopted by both academia and industry. His work is situated at the intersection of machine learning systems and applied mathematics, drawing on techniques from numerical linear algebra and optimization.
Education and Early Career
Gholami received his Bachelor of Science degree in Electrical Engineering from the Sharif University of Technology in Tehran, Iran, in 2011. He then moved to the United States for graduate studies, earning a Master of Science degree in Electrical and Computer Engineering from the University of Texas at Austin in 2013. He completed his Ph.D. in Computational and Applied Mathematics at the University of Texas at Austin in 2017, under the supervision of George Biros. His doctoral dissertation focused on scalable algorithms for solving partial differential equations on high-performance computing systems, which laid the groundwork for his later interest in efficient machine learning.
During his Ph.D., Gholami worked on parallel numerical algorithms for problems in computational fluid dynamics and medical imaging. This experience gave him a deep understanding of hardware-software co-design and the importance of memory access patterns in achieving high performance, skills that would later prove crucial in his AI research.
Postdoctoral Research at UC Berkeley
From 2017 to 2020, Gholami was a postdoctoral researcher at the BAIR (Berkeley AI Research) laboratory at the University of California, Berkeley, where he worked with Kurt Keutzer and Michael Mahoney. This period was formative for his shift toward efficient deep learning. At Berkeley, he became involved in the development of the Deep Compression framework and contributed to the design of the SqueezeNet architecture, which demonstrated that convolutional neural networks could achieve high accuracy with dramatically fewer parameters.
Gholami's postdoctoral work also explored the theoretical foundations of quantization. He published influential papers on the effects of different quantization schemes on model accuracy, including a systematic study of post-training quantization techniques. This research helped establish best practices for converting full-precision models to lower-bit representations without significant loss in performance.
Contributions to Efficient Deep Learning
Gholami's most significant contributions lie in the area of model compression and acceleration. He has developed methods for reducing the memory footprint and computational cost of deep neural networks, making them deployable on mobile phones, embedded systems, and other resource-constrained platforms.
One of his notable works is the development of a framework for mixed-precision quantization, which automatically determines the optimal bit-width for each layer of a neural network. This approach, based on solving a constrained optimization problem, can achieve up to 4x compression with minimal accuracy degradation compared to uniform quantization. The method has been adopted in several commercial toolchains for edge AI.
Gholami has also contributed to the understanding of the relationship between model size and training dynamics. His research on the "right scaling" of learning rates for large-batch training has been widely cited, providing practical guidance for distributed training on clusters of GPUs. He showed that the learning rate should be scaled linearly with batch size up to a certain point, after which more sophisticated schedules are needed.
Work on Large Language Models
With the rise of Large language models and Transformer (architecture) architectures, Gholami turned his attention to the unique challenges posed by these models. He has investigated techniques for quantizing the weights and activations of transformers, which are particularly sensitive to numerical precision due to the presence of outlier values in activations. His work on outlier-aware quantization has been instrumental in enabling the deployment of models like GPT and BERT on consumer hardware.
Gholami has also explored methods for reducing the memory overhead of attention mechanisms, which scale quadratically with sequence length. He contributed to the development of sparse attention patterns and low-rank approximations that maintain accuracy while reducing computational complexity. These techniques are relevant for processing long documents, genomic sequences, and other data with extended context.
In addition, Gholami has written comprehensive surveys on efficient training and inference of large language models, synthesizing the state of the art in areas such as pruning, distillation, and hardware-aware design. These surveys have become essential reading for researchers entering the field.
Academic Appointments and Industry Roles
In 2020, Gholami joined the faculty of the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley, as an Assistant Professor. He leads the Efficient and Scalable AI Lab, where he supervises graduate students working on topics ranging from low-bit quantization to federated learning. He has received several awards for his teaching and research, including the NSF CAREER Award in 2022.
Gholami has also maintained close ties with industry. He has held visiting researcher positions at Google DeepMind and Samsung Research, where he collaborated on projects related to on-device AI. He has served as a consultant for several startups focusing on edge inference, and his open-source libraries, such as the Deep Compression Toolkit, are used by companies including Intel and Qualcomm for optimizing their neural network accelerators.
Selected Publications and Impact
Gholami has authored over 80 peer-reviewed papers, accumulating more than 20,000 citations as of 2024. Some of his most influential works include:
- "A Survey of Quantization Methods for Efficient Neural Network Inference" (2021), which provides a comprehensive taxonomy of quantization techniques and has been cited over 1,500 times.
- "Mixed-Precision Quantization of CNNs via Limited Range Parameterization" (2021), introducing a novel parameterization that improves the trade-off between bit-width and accuracy.
- "Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models" (2022), a paper that addresses the challenge of activation outliers in transformers.
His research has been recognized with best paper awards at the International Conference on Learning Representations (ICLR) and the Conference on Computer Vision and Pattern Recognition (CVPR). He has also been invited to give keynote talks at major industrial conferences, including the AI Hardware Summit and the Efficient Deep Learning Workshop.
Teaching and Mentorship
At Berkeley, Gholami teaches graduate courses on efficient machine learning and parallel computing. He is known for his hands-on approach, encouraging students to implement algorithms from scratch and benchmark them on real hardware. Several of his Ph.D. students have gone on to research positions at leading AI labs such as OpenAI and Anthropic.
Gholami is also an advocate for reproducible research. He has made the code and data for most of his papers publicly available, and he maintains a blog where he explains complex topics in accessible language. He has organized multiple workshops on efficient deep learning at NeurIPS and ICML, fostering a community of researchers dedicated to sustainable AI.
Current Research Directions
As of 2025, Gholami's research focuses on three main areas: (1) the development of quantization-aware training methods for large language models, (2) the design of energy-efficient hardware-software co-optimization strategies for edge AI, and (3) the application of efficient deep learning to scientific computing, such as solving partial differential equations with neural operators.
He is also involved in projects that aim to reduce the carbon footprint of AI training. His work on adaptive precision and early-exit mechanisms has shown potential to cut energy consumption by up to 30% without sacrificing model quality. Gholami collaborates with hardware vendors like AMD and Arm Holdings to ensure that his algorithms align with the capabilities of next-generation processors.
Recognition and Awards
Gholami has received numerous honors throughout his career. In addition to the NSF CAREER Award, he was named a Sloan Research Fellow in 2023. He has been listed among the "AI 2000 Most Influential Scholars" in the field of machine learning for three consecutive years. His contributions to open-source software were recognized with the Google Open Source Peer Bonus Award in 2021.
He serves on the program committees of major conferences and is an associate editor for the IEEE Transactions on Pattern Analysis and Machine Intelligence. His work has been featured in popular media outlets, including articles in IEEE Spectrum and MIT Technology Review, highlighting the practical impact of his methods on real-world AI deployment.
Personal Life and Interests
Outside of research, Gholami is an avid hiker and photographer. He often combines these interests during his travels to academic conferences, capturing landscapes and urban scenes. He is also a mentor for first-generation college students and participates in outreach programs to encourage underrepresented groups to pursue careers in STEM.
Gholami maintains an active presence on social media, where he shares insights about recent papers and provides career advice to aspiring researchers. He is known for his collaborative spirit and willingness to engage with junior researchers, often co-authoring papers with students from other universities.
Legacy and Future Outlook
The field of efficient deep learning has grown rapidly, and Gholami is considered one of its leading figures. His work has not only advanced academic knowledge but also enabled practical applications in areas such as autonomous driving, medical imaging, and real-time language translation. As AI models continue to grow in size, the importance of efficiency research is likely to increase, and Gholami's contributions will remain relevant for years to come.
He is currently exploring the intersection of efficient deep learning and Generative AI, aiming to make diffusion models and other generative architectures more accessible on mobile devices. His long-term vision is to create AI systems that are not only powerful but also sustainable and inclusive, capable of running on the billions of devices already in use around the world.