# Gaudi (processor)

Gaudi is a family of AI accelerators developed by Intel for deep learning training and inference, featuring heterogeneous compute and high-bandwidth memory. The first generation launched in 2019, with Gaudi 2 and Gaudi 3 following in 2022 and 2024.

**Gaudi** is a family of application-specific integrated circuits (ASICs) designed by Intel for accelerating [deep learning](https://www.wikiprompt.org/wiki/deep-learning) workloads, particularly [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) training and inference. The architecture emphasizes a heterogeneous compute approach, integrating multiple tensor processing cores with a dedicated network-on-chip and high-bandwidth memory (HBM) to enable efficient scaling across multiple accelerators. Gaudi processors are positioned as competitors to [Nvidia](https://www.wikiprompt.org/wiki/nvidia)'s GPU-based accelerators and other AI chips such as [AMD](https://www.wikiprompt.org/wiki/amd)'s Instinct series and [Google Cloud](https://www.wikiprompt.org/wiki/google-cloud)'s TPUs.

The first-generation Gaudi processor, announced in 2019, was developed by Habana Labs, an Israeli AI chip startup acquired by Intel in 2019 for approximately $2 billion. The second generation, Gaudi 2, launched in 2022, and the third generation, Gaudi 3, was unveiled in 2024. Intel markets Gaudi as a cost-effective alternative for AI training and inference, with a focus on open software ecosystems and support for popular frameworks like [PyTorch](https://www.wikiprompt.org/wiki/pytorch) and [TensorFlow](https://www.wikiprompt.org/wiki/tensorflow).

## History

Gaudi's origins trace to Habana Labs, founded in 2016 by David Dahan, Ran Halutz, and others, with headquarters in Caesarea, Israel. The company initially developed the Goya inference chip, released in 2018, before shifting focus to training with Gaudi. Intel acquired Habana Labs in December 2019, integrating the team into its Data Platforms Group. The acquisition was part of Intel's broader strategy to compete in the AI accelerator market, which had been dominated by Nvidia's GPUs.

The first Gaudi processor, Gaudi 1, was officially launched in 2019, with early customers including [Amazon Web Services](https://www.wikiprompt.org/wiki/amazon-web-services) (AWS), which offered Gaudi-based instances in 2021. Gaudi 2, announced in May 2022, featured significant improvements in memory capacity and compute performance, and was adopted by cloud providers and enterprises. In April 2024, Intel introduced Gaudi 3, which incorporated a more advanced 5nm process (manufactured by [TSMC](https://www.wikiprompt.org/wiki/tsmc)) and doubled performance over its predecessor for large language model training.

## Architecture

Gaudi processors are built around a heterogeneous architecture that combines multiple tensor processing cores (TPCs) with a scalable network fabric. Each Gaudi chip includes a set of fully programmable TPCs that execute matrix multiplications and other tensor operations, similar to the tensor cores in Nvidia GPUs. The TPCs are complemented by a dedicated memory subsystem using HBM, which provides high bandwidth for feeding data to the compute units.

A distinctive feature of Gaudi is its integrated networking capability. Unlike GPUs that typically require external InfiniBand or Ethernet switches for multi-node scaling, Gaudi includes on-chip RDMA (Remote Direct Memory Access) over Ethernet (RoCE) engines. This allows multiple Gaudi accelerators to be interconnected directly, reducing the cost and complexity of building large-scale AI clusters. The network-on-chip (NoC) enables efficient communication between TPCs within a single chip, while the RoCE engines handle inter-chip communication.

Gaudi 2 and Gaudi 3 also incorporate a dedicated matrix multiplication engine, separate from the TPCs, to accelerate operations common in transformers and other modern neural network architectures. The processors support mixed-precision training, including FP32, BF16, and FP8 formats, to balance accuracy and performance.

## Generations

### Gaudi 1

The first-generation Gaudi processor, released in 2019, featured 32 TPCs and 16 GB of HBM2 memory with a bandwidth of 1.2 TB/s. It was designed for training and inference of [neural network](https://www.wikiprompt.org/wiki/neural-network) models, with a focus on efficiency and scalability. Gaudi 1 achieved a performance of 1.6 TFLOPS per chip in BF16 precision (though this figure is often cited as 1.6 TFLOPS for dense matrix operations). The chip was manufactured using a 16nm process from TSMC.

### Gaudi 2

Gaudi 2, launched in 2022, doubled the number of TPCs to 64 and increased HBM capacity to 96 GB of HBM2e, with a bandwidth of 2.4 TB/s. It also added a dedicated matrix multiplication engine, boosting performance to 3.2 TFLOPS in BF16. Gaudi 2 was built on a 7nm process. Intel claimed that Gaudi 2 offered superior price-performance compared to Nvidia's A100 GPU for training large language models, citing benchmarks on models like [GPT-3](https://www.wikiprompt.org/wiki/gpt-3) and [BERT](https://www.wikiprompt.org/wiki/bert).

### Gaudi 3

Gaudi 3, announced in April 2024, represented a major architectural leap. It integrated 128 TPCs and a more powerful matrix engine, with 128 GB of HBM2e memory and a bandwidth of 3.7 TB/s. The chip was manufactured on a 5nm process at TSMC. Intel stated that Gaudi 3 could train a 175-billion-parameter [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) (similar to GPT-3) 50% faster than Nvidia's H100 GPU, while consuming less power. Gaudi 3 also introduced support for FP8 precision, further improving throughput.

## Software Ecosystem

Gaudi processors are supported by Intel's SynapseAI software stack, which provides a compiler, runtime, and libraries optimized for the hardware. SynapseAI integrates with popular deep learning frameworks, including [PyTorch](https://www.wikiprompt.org/wiki/pytorch) and [TensorFlow](https://www.wikiprompt.org/wiki/tensorflow), allowing developers to run models with minimal code changes. Intel also provides a [Hugging Face](https://www.wikiprompt.org/wiki/hugging-face) integration, enabling easy deployment of pre-trained transformers.

The software stack includes tools for model optimization, such as quantization and graph compilation, as well as support for distributed training across multiple Gaudi nodes. Intel has emphasized openness, with Gaudi supporting standard networking protocols (RoCE) and being compatible with Kubernetes for orchestration in cloud environments.

## Performance and Benchmarks

Intel has published benchmark results for Gaudi processors, particularly for training and inference of large language models. In 2023, Intel claimed that Gaudi 2 achieved 1.5x to 2x better price-performance than Nvidia's A100 for training GPT-3 and [Llama 2](https://www.wikiprompt.org/wiki/llama-2) models. For Gaudi 3, Intel reported a 1.7x improvement in training throughput for GPT-3 compared to Nvidia's H100, and a 2x improvement for inference on models like [Falcon](https://www.wikiprompt.org/wiki/falcon) and [Llama 3](https://www.wikiprompt.org/wiki/llama-3).

Independent benchmarks have been limited, but some cloud providers, such as AWS and IBM Cloud, have offered Gaudi-based instances and reported competitive results. In 2024, [Intel](https://www.wikiprompt.org/wiki/intel) also announced a collaboration with [Google Cloud](https://www.wikiprompt.org/wiki/google-cloud) to offer Gaudi 3 as a service, expanding its availability.

## Market Position and Competition

Gaudi competes directly with Nvidia's data center GPUs, including the A100, H100, and newer H200, as well as with [AMD](https://www.wikiprompt.org/wiki/amd)'s Instinct MI300 series and [AWS](https://www.wikiprompt.org/wiki/amazon-web-services)'s Trainium and Inferentia chips. Intel positions Gaudi as a more cost-effective alternative, particularly for enterprises and cloud providers looking to reduce AI infrastructure costs. The company has also highlighted Gaudi's energy efficiency, which is a growing concern for data centers.

Despite its technical merits, Gaudi has faced challenges in gaining widespread adoption, partly due to Nvidia's dominant software ecosystem (CUDA) and established market presence. Intel has attempted to counter this by supporting open standards and providing migration tools, but as of 2025, Gaudi's market share remains small relative to Nvidia.

## Applications

Gaudi processors are used for both training and inference of deep learning models, with a focus on large language models, computer vision, and recommendation systems. They are deployed in cloud data centers, on-premises enterprise systems, and edge environments. Key use cases include natural language processing, image recognition, and generative AI applications such as chatbots and content generation.

In addition to cloud services, Intel has partnered with system integrators to offer Gaudi-based servers and workstations. The processors are also used in research settings, with universities and AI labs leveraging Gaudi for model development.

## Reception and Criticism

Industry analysts have generally viewed Gaudi as a credible alternative to Nvidia, but note that software maturity and ecosystem support remain weaker. Some critics have pointed out that Intel's benchmark claims are often based on specific configurations that may not reflect real-world performance. Additionally, the rapid pace of Nvidia's product releases (e.g., H100, H200, and the upcoming Blackwell architecture) has made it difficult for Gaudi to maintain a performance edge.

Intel has responded by emphasizing total cost of ownership (TCO) and power efficiency, arguing that Gaudi offers better value for large-scale deployments. The company has also committed to a regular release cadence, with Gaudi 4 expected in 2025.

## Future Developments

Intel has announced plans to continue developing the Gaudi family, with Gaudi 4 expected to use a more advanced process and further improve performance. The company is also working on integrating Gaudi with its Xeon processors and oneAPI unified programming model to simplify heterogeneous computing. As of 2025, Intel aims to capture a larger share of the AI accelerator market, targeting both cloud and enterprise segments.

## See Also

- AI accelerator
- [Deep learning](https://www.wikiprompt.org/wiki/deep-learning)
- [Large language model](https://www.wikiprompt.org/wiki/large-language-model)
- [Nvidia](https://www.wikiprompt.org/wiki/nvidia)
- [AMD](https://www.wikiprompt.org/wiki/amd)
- [AWS Trainium](https://www.wikiprompt.org/wiki/aws-trainium)

## References

1. Intel Newsroom. "Intel Acquires Habana Labs to Expand AI Portfolio." December 16, 2019.
2. Habana Labs. "Gaudi Training Processor." 2019.
3. Intel. "Intel Gaudi 2 AI Accelerator." 2022.
4. Intel. "Intel Gaudi 3 AI Accelerator." 2024.
5. Cloud provider documentation for Gaudi instances (AWS, IBM Cloud, Google Cloud).

---
Source: https://www.wikiprompt.org/wiki/intel-gaudi
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-10T03:23:24.324753+00:00