Jurassic is a family of large language models developed by AI21 Labs, an Israeli artificial intelligence company founded in 2017. The Jurassic series is among the earliest commercial large-scale transformer-based models, positioned as a competitor to models from OpenAI and Anthropic. The family includes several generations, from the original Jurassic-1 released in 2021 to the more recent Jurassic-2 and specialized variants, all designed for natural language understanding and generation tasks such as text completion, summarization, and question answering.
The models are built on the Transformer (architecture) architecture, a Deep learning framework introduced in 2017 that underpins most modern generative AI systems. AI21 Labs trained Jurassic models on extensive text corpora, using neural network techniques and optimization methods like Adam and learning rate scheduling. Jurassic models are available through AI21 Labs' proprietary API, as well as on cloud platforms including Amazon Web Services (AWS), Microsoft Azure, and Google Cloud, making them accessible to developers and enterprises.
History and Development
AI21 Labs announced Jurassic-1 in August 2021, with two model sizes: J1-Jumbo (178 billion parameters) and J1-Large (7.5 billion parameters). At the time, J1-Jumbo was among the largest language models publicly available, comparable to OpenAI's GPT-3, which had 175 billion parameters. The development team included AI21 co-founders Eilon Reshef and Dafna Sharon, along with researchers who had backgrounds in Machine learning and computational linguistics.
In 2022, AI21 Labs released Jurassic-2, which introduced improvements in instruction following and reasoning. The Jurassic-2 family included models such as J2-Jumbo, J2-Grande, and J2-Mid, with parameter counts ranging from 7.8 billion to 178 billion. These models were optimized for specific use cases, including generative AI applications in customer service, content creation, and code generation.
By 2023, AI21 Labs expanded the Jurassic line with specialized models. In October 2023, the company released Jurassic-2 Ultra, a 178-billion-parameter model with enhanced multilingual capabilities. The following year, AI21 introduced Jurassic-2 Mid and Jamba, the latter being a hybrid architecture combining transformer layers with state space models, which improved efficiency for long-context tasks.
Architecture and Training
Jurassic models are based on the Transformer (architecture) architecture, specifically the decoder-only variant, which is also used by OpenAI's GPT series. The models employ multi-head attention mechanisms to process input sequences, along with positional encodings to capture word order. Training uses Cross-Attention in some variants, though the core architecture is autoregressive, predicting the next token in a sequence.
AI21 Labs trained Jurassic models using large-scale distributed computing resources, likely on NVIDIA GPUs, though specific hardware details are not publicly documented. The training process involved data augmentation and curriculum learning strategies to improve generalization. Optimization relied on stochastic gradient descent variants such as Adam, with gradient clipping and layer normalization to stabilize training.
The models use top-k and top-p sampling methods for text generation, along with temperature scaling to control output randomness. Beam search is available for tasks requiring more deterministic outputs, such as translation or summarization.
Capabilities and Features
Jurassic models support a wide range of natural language processing tasks. They can perform text completion, summarization, paraphrasing, and question answering. The models also handle code generation and completion, making them useful for software development workflows. Jurassic-2 and later versions include instruction-following capabilities, allowing users to specify tasks in natural language without fine-tuning.
A notable feature is the support for multiple languages. Jurassic-2 models were trained on data in over 100 languages, with particular strength in English, Spanish, French, German, and Hebrew, reflecting AI21 Labs' Israeli origins. The models also support structured inputs, such as JSON and SQL, enabling integration with databases and APIs.
AI21 Labs provides a developer platform with an API that includes features like custom model fine-tuning, semantic search, and a fact-checking tool called "Wordtune Read" and "Wordtune Write," which are separate products but share underlying technology.
Comparison with Other Models
Jurassic models compete directly with OpenAI's GPT-3 and GPT-4, Anthropic's Claude, and Google DeepMind's Gemini. In benchmark evaluations, Jurassic-1 Jumbo performed comparably to GPT-3 on tasks like SuperGLUE and LAMBADA, though it lagged on some reasoning benchmarks. Jurassic-2 improved on these scores, particularly in multi-turn dialogue and instruction following.
A key differentiator is AI21 Labs' focus on enterprise applications. The company emphasizes data privacy and offers on-premises deployment options, which appeal to organizations with strict compliance requirements. Additionally, Jurassic models are available on AWS through Amazon Bedrock, on Azure through Microsoft's AI services, and on Google Cloud Vertex AI, providing multiple integration paths.
Applications and Use Cases
Jurassic models are used in various industries. In customer service, they power chatbots and virtual assistants that handle inquiries and resolve issues. In content creation, they assist with drafting articles, marketing copy, and social media posts. Legal and financial firms use Jurassic for document summarization and contract analysis.
AI21 Labs also offers specialized tools built on Jurassic, such as Wordtune, an AI-powered writing assistant that suggests rephrasing and tone adjustments. The company has partnered with Samsung to integrate AI features into mobile devices, and with Intel for hardware optimization.
Performance and Benchmarks
Jurassic models have been evaluated on standard NLP benchmarks. On the SuperGLUE benchmark, J1-Jumbo achieved a score of 82.4, slightly below GPT-3's 83.5. On the LAMBADA language modeling test, J1-Jumbo reached 82.5% accuracy. Jurassic-2 models showed improvements in few-shot learning, with J2-Jumbo outperforming GPT-3 on several tasks in the MMLU benchmark.
In 2023, AI21 Labs reported that Jurassic-2 Ultra achieved state-of-the-art results on the HELM benchmark for some tasks, though independent verification is limited. The company publishes performance metrics on its website, but detailed evaluation methodologies are not always disclosed.
Availability and Pricing
Jurassic models are accessible through AI21 Labs' API, which offers a pay-as-you-go pricing model based on tokens processed. As of 2024, pricing starts at $0.01 per 1,000 tokens for the smallest models and scales up for larger ones. The API supports both synchronous and asynchronous requests, with a maximum context length of 8,192 tokens for most models.
AI21 Labs also provides a free tier for developers to experiment with the models. Enterprise customers can negotiate custom contracts, including dedicated infrastructure and service-level agreements. The models are also available through cloud marketplaces, such as AWS Marketplace and Azure Marketplace.
Future Directions
AI21 Labs continues to develop the Jurassic family, with a focus on improving efficiency and reducing computational costs. The company has explored model pruning and residual network techniques to create smaller, faster models without significant performance loss. In 2024, AI21 announced research into hybrid architectures that combine transformer layers with recurrent components, aiming to handle longer sequences more efficiently.
The company is also investing in alignment and safety, using methods like reinforcement learning from AI feedback to reduce harmful outputs. As the field of generative AI evolves, Jurassic models are expected to integrate more multimodal capabilities, potentially processing images and audio in addition to text.
Reception and Impact
Jurassic models have been well-received in the AI community for their strong performance and accessibility. AI21 Labs has raised significant funding, including a $208 million Series C round in 2022, valuing the company at over $1.4 billion. The models have been used in academic research and industry applications, contributing to the broader adoption of large language models outside the dominant US tech companies.
Critics note that Jurassic models have not achieved the same level of public recognition as GPT-4 or Claude, partly due to less aggressive marketing. However, the models are respected for their multilingual capabilities and enterprise focus, making AI21 Labs a notable player in the competitive AI landscape.