Jais is an open-source Large language model designed to generate high-quality text in both Arabic and English. Launched in August 2023, it was developed through a collaboration between the Emirati AI company G42, the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), and the US-based hardware company Cerebras Systems. The model was created to address the underrepresentation of the Arabic language in generative artificial intelligence, aiming to serve the world's 400 million Arabic speakers with culturally and linguistically accurate outputs. Its name references Jebel Jais, the highest mountain in the United Arab Emirates.
Background and development
Jais emerged from the recognition that existing large language models often performed poorly on Arabic due to limited or low-quality training data. The project represents a significant investment by the UAE in Artificial intelligence as part of its national strategy. The development partnership involved Inception (now Core42), a subsidiary of G42; MBZUAI, a research university dedicated to AI; and Cerebras Systems, which provided specialized AI hardware. The collaboration aimed to create a model that could handle Arabic's diverse dialects and cultural nuances, which were often neglected in models trained predominantly on English data.
Training
The initial version of Jais, released in August 2023, had 13 billion parameters. In November 2023, Core42 released an improved version, Jais 30B, with 30 billion parameters. Both models were trained on a subset of the Cerebras Condor Galaxy 1 supercomputer, a powerful computing system designed for large-scale Deep learning tasks. The training dataset comprised a mix of Arabic, English, and computer code. According to Timothy Baldwin, a professor of natural language processing at MBZUAI, training on a diverse Arabic dataset allows the model to switch between dialects, enhancing its practical utility for Arabic speakers.
Features
Jais is designed to generate text in both English and Arabic, making it a bilingual tool for a wide range of applications. The project has also released instruction-tuned "Chat" variants for both the 13B and 30B models, which are specifically optimized for conversational use cases. These variants are fine-tuned to follow user instructions and engage in dialogue, similar to other Transformer (architecture)-based models. Future releases are planned to include additional functionality for working with images, graphs, and tabular data, expanding the model's capabilities beyond text generation.
Impact and significance
Jais contributes to the diversification of the Large language model landscape, which has been dominated by English-centric models from companies like OpenAI and Google DeepMind. By focusing on Arabic, Jais helps bridge the gap in AI accessibility for a large population that has been underserved by existing technologies. The open-source nature of the model allows researchers and developers to adapt and improve it, fostering innovation in Arabic Natural language processing and related fields. As of its release, Jais represented a notable step toward more inclusive AI, though its long-term impact will depend on community adoption and continued development.
References
External links
Official website