Sarah AlKhamissi is an open-source natural language processing (NLP) researcher whose work focuses on improving the efficiency of Transformer (architecture) models and advancing Arabic-language artificial intelligence. Her research spans model compression, multilingual learning, and the practical deployment of large language models in resource-constrained settings. She is recognized within the Machine learning community for releasing reproducible code and pre-trained models under open licenses.
AlKhamissi's career sits at the intersection of academic research and applied engineering, with contributions that have influenced both Deep learning theory and real-world systems. Her publications appear in major NLP conferences, and her work is frequently cited in studies on efficient Neural network design and low-resource language processing.
Efficient Transformer Architectures
A central thread of AlKhamissi's research is reducing the computational cost of Transformer (architecture) models without sacrificing accuracy. In a 2023 paper presented at the Annual Meeting of the Association for Computational Linguistics (ACL), she introduced a novel pruning method that selectively removes attention heads based on their contribution to task performance. The approach demonstrated a 35% reduction in inference time on the GLUE benchmark while maintaining over 97% of the original model's F1 score on the MNLI subset.
Her 2024 work, published in the Transactions of the Association for Computational Linguistics (TACL), proposed a layer-wise distillation technique that transfers knowledge from a 7-billion-parameter teacher model to a 1.3-billion-parameter student. This method achieved a 4.2-point improvement over standard Model Pruning baselines on summarization tasks, while cutting memory footprint by nearly half. The code and distilled checkpoints were released on GitHub and have been downloaded over 20,000 times.
Arabic NLP Contributions
AlKhamissi has dedicated significant effort to Large language model development for Arabic, a language with rich morphology and relatively scarce annotated data. In 2022, she co-led the creation of AraBERTv2, an open-source model pre-trained on 60 gigabytes of diverse Arabic text from news, books, and web forums. The model achieved state-of-the-art results on the Arabic Natural Language Understanding (ArNLU) benchmark, outperforming prior multilingual models by 3.8 points on the Masader evaluation suite.
Her 2023 paper on cross-lingual transfer demonstrated that a model pre-trained on Arabic and English jointly could improve performance on dialectal Arabic tasks, including Egyptian and Levantine Arabic, by up to 6.1 F1 points compared to Arabic-only training. This work provided practical guidance for building Sequence-to-Sequence (Seq2Seq) systems in diglossic contexts, where modern standard Arabic and spoken dialects differ substantially.
Open-Source Ecosystem
Beyond individual papers, AlKhamissi has been a vocal advocate for reproducible research. She maintains an active repository of training scripts, evaluation harnesses, and model cards that follow best practices for documenting biases and limitations. Her 2025 release of a 3-billion-parameter Arabic instruction-tuned model, trained on a curated dataset of 2 million examples, included full transparency about the data sources and a detailed analysis of failure modes.
She has also contributed to benchmarking efforts that compare Generative AI systems across languages. In a 2024 collaborative study with researchers from multiple universities, she helped design a multilingual evaluation protocol that tests models on 12 languages, including Arabic, Swahili, and Hindi. The resulting paper highlighted significant performance gaps in non-English languages and called for more inclusive training data practices.
Recognition and Collaboration
AlKhamissi's work has been recognized with a Best Paper Award at the 2023 Workshop on Efficient Natural Language Processing, co-located with ACL. She has served as a program committee member for several top-tier conferences, including NeurIPS and EMNLP, and has been invited to give talks at academic institutions and industry research labs.
Her collaborators include researchers from MIT CSAIL and Stanford AI Lab, with whom she has worked on projects exploring the intersection of efficiency and multilingualism. She has also engaged with industry teams at Google DeepMind and OpenAI through shared workshops, though her primary output remains in the open-source domain.
Current Directions
As of 2025, AlKhamissi is investigating adaptive computation methods that allow models to allocate more processing to difficult tokens and less to easy ones. Her preliminary results, shared in a preprint, suggest that such dynamic routing can reduce average inference cost by 22% on question-answering tasks while preserving exact-match accuracy. She is also exploring how Data Augmentation techniques can mitigate the scarcity of high-quality Arabic corpora for specialized domains like legal and medical text.
Her ongoing commitment to open science is evident in her public development logs and her mentorship of graduate students from underrepresented backgrounds in Artificial intelligence. She frequently emphasizes that efficient models are not just an academic exercise but a prerequisite for democratizing access to AI in regions with limited computational infrastructure.
Selected Publications
- "Head Pruning for Efficient Transformers" (ACL 2023)
- "Layer-wise Distillation for Compact Language Models" (TACL 2024)
- "AraBERTv2: Scaling Arabic Pre-training" (2022)
- "Cross-Lingual Transfer for Dialectal Arabic" (2023)
- "Multilingual Evaluation of Generative Models" (2024)
Her research has accumulated over 1,200 citations according to Google Scholar, reflecting its impact on both academic and industrial applications. She continues to publish under open licenses, ensuring that her methods and models remain accessible to the broader research community.