NVLink

NVLink is a wire-based serial, multi-lane, near-range communications link developed by Nvidia for data and control code transfers between CPUs and GPUs and between GPUs. Unlike PCI Express, a device can consist of multiple NVLinks, and devices can use mesh networking to communicate instead of a central hub or switch. The protocol was first announced in March 2014 and uses a proprietary high-speed signaling interconnect (NVHS). NVLink is designed to address the growing bandwidth demands of high-performance computing, artificial intelligence, and machine learning workloads, providing a low-latency, high-throughput alternative to traditional bus architectures.

Architecture and Versions

NVLink specifies a point-to-point connection with data rates of 20, 25, and 50 Gbit/s per differential pair for versions 1.0, 2.0, and 3.0 and higher, respectively. For NVLink 1.0 and 2.0, eight differential pairs form a sub-link, and two sub-links (one for each direction) form a link. Starting with NVLink 3.0, only four differential pairs form a sub-link. For NVLink 2.0 and higher, the total data rate for a sub-link is 25 GB/s, and the total data rate for a link is 50 GB/s. Each V100 GPU supports up to six links, providing up to 300 GB/s total bidirectional bandwidth. Announced on May 14, 2020, NVLink 3.0 increased the per-pair data rate to 50 Gbit/s while halving the number of pairs per link. With 12 links on the Ampere-based A100 GPU, total bandwidth reached 600 GB/s. The Hopper microarchitecture, announced in March 2022, features 18 NVLink 4.0 links, enabling 900 GB/s. NVLink 6, used in Nvidia's Vera Rubin NVL72 platform, provides 3.6 TB/s of bidirectional bandwidth per GPU, with nine NVLink 6 switches delivering 260 TB/s of total scale-up bandwidth per rack.

Scalability and Switching

For small numbers of GPUs, the NVLink lanes on a single device are sufficient for all-to-all mesh connectivity. To accommodate higher GPU counts, NVLink has used a packet-switched architecture since 2018, where a central switch can serve up to 32 two-lane ports. The NVSwitch for NVLink 4.0 can perform simple computations (e.g., sum, broadcast) to reduce communication overhead via the SHARP accelerator. This switch-based approach enables building large GPU clusters, such as those used in deep learning training and large language model inference, where efficient scaling is critical.

Performance Characteristics

Real-world performance is affected by overheads such as 128b/130b line coding, link control characters, transaction headers, buffering capabilities, and DMA usage on the host side. These factors typically reduce the data rate to 90-95% of the theoretical transfer rate. Benchmarks show an achievable transfer rate of about 35.3 Gbit/s (host to device) for a 40 Gbit/s NVLink connection towards a P100 GPU in a system driven by IBM POWER8 CPUs. The table below compares basic metrics across versions:

Use with Plug-in Boards

NVLink is also available on certain high-end graphics cards, which feature physical connectors for joining cards into an NVLink group. These connectors are typically U-shaped with fine grid edge connectors, and the width determines the slot spacing required (commonly 3 to 5 slots). The interconnect is often referred to as Scalable Link Interface (SLI) due to its structural design, though the modern NVLink is technically different. Reported devices include Quadro GP100 (up to 160 GB/s with two bridges), Quadro GV100 (up to 200 GB/s), GeForce RTX 2080 and 2080 Ti (single bridge), GeForce RTX 3090 (unique bridge), and Quadro RTX 5000/6000/8000 (up to 100 GB/s).

Software and Programming

For Tesla, Quadro, and Grid product lines, the NVML-API (Nvidia Management Library API) provides functions for controlling NVLink interconnects on Windows and Linux, including component evaluation, status/error querying, and performance monitoring. The NCCL library (Nvidia Collective Communications Library) enables developers to implement powerful AI and computation-heavy applications atop NVLink. The Nvidia Control Panel's "3D Settings" page and the CUDA sample "simpleP2P" use these APIs. On Linux, the command-line tool nvidia-smi nvlink offers similar advanced information and control.

History

On April 5, 2016, Nvidia announced that NVLink would be implemented in the Pascal-based GP100 GPU, used in products like the Tesla P100. The DGX-1 high-performance computer allowed up to eight P100 modules in a single rack, connected to up to two host CPUs via a dedicated carrier board for routing NVLink connections. Each P100 required 800 pins (400 for PCIe and power, and another 400 for NVLink). Subsequent generations (Volta, Ampere, Hopper, and later) have continued to increase link counts and bandwidth, solidifying NVLink as a key technology for generative AI and high-performance computing.