Arm AI is the artificial intelligence-focused division of Arm Holdings plc, a British semiconductor and software design company headquartered in Cambridge, England. Arm AI develops processor designs and neural processing units (NPUs) tailored for edge computing, enabling on-device Machine learning and Deep learning workloads in devices ranging from smartphones to embedded sensors. Its Ethos NPU family is a key product line, designed to accelerate Neural network inference efficiently in power-constrained environments.
The division builds on Arm's broader architecture, which is licensed to partners worldwide. Arm's CPU cores, such as the Cortex-A and Cortex-M series, are widely used in mobile and embedded systems, and Arm AI integrates AI-specific accelerators with these cores. This approach allows devices to run Generative AI models and other AI tasks locally, reducing reliance on cloud computing and enhancing privacy and latency. Arm AI competes with other edge AI solutions from companies like Qualcomm, Apple, and Samsung Electronics, but its licensing model differentiates it by enabling a wide ecosystem of chip designers.
History and Evolution
Arm Holdings was founded on 12 November 1990 as Widelogic Limited, quickly renamed Advanced RISC Machines Limited on 3 December 1990, as a joint venture between Acorn Computers, Apple, and VLSI Technology. The acronym ARM originally stood for "Acorn RISC Machine," later changed to "Advanced RISC Machines" at Apple's request. The company's first processor, the ARM1, was used in the Acorn Archimedes, one of the first RISC-based desktop computers. Arm's early focus was on low-power, high-efficiency processors, which became foundational for mobile devices.
Arm AI emerged as a distinct initiative in the 2010s as Artificial intelligence applications proliferated. In 2017, Arm introduced its first AI-specific processor architecture, the Arm Machine Learning processor, and in 2018 launched the Ethos NPU product line. These developments aimed to bring AI capabilities to billions of edge devices, aligning with Arm's mission to compute everywhere. The division has since expanded to support Large language model inference on-device, a growing demand as models like those from OpenAI and Anthropic become more compact.
Ethos NPU Architecture
The Ethos NPU family is Arm AI's core hardware offering. The first generation, Ethos-N37, was announced in 2018, targeting mid-range smartphones and IoT devices. Subsequent generations, such as Ethos-N57 and Ethos-N77, scaled performance for higher-end applications. In 2020, Arm introduced the Ethos-U series, designed for microcontrollers and ultra-low-power edge devices, enabling AI in sensors and wearables. The latest Ethos-U85, released in 2024, supports transformer-based models, including Transformer (architecture) architectures used in modern AI.
Ethos NPUs are optimized for inference, not training, focusing on efficiency metrics like TOPS/W (tera-operations per second per watt). They support common neural network operations, including convolution, pooling, and activation functions, and are programmable via software frameworks like TensorFlow Lite and PyTorch. Arm AI also provides the Arm NN software library, which bridges AI frameworks to the hardware, ensuring seamless deployment.
Integration with Arm CPUs
Arm AI processors are designed to work in tandem with Arm CPUs, which are ubiquitous in mobile and embedded systems. The Cortex-A series, used in smartphones, and Cortex-M series, used in microcontrollers, provide general-purpose compute, while Ethos NPUs offload AI-specific tasks. This heterogeneous computing approach balances performance and power, critical for battery-operated devices. For example, a smartphone might use a Cortex-A78 for app processing and an Ethos-N77 for camera AI features like scene recognition.
Arm's Total Compute solutions, introduced in 2020, integrate CPUs, GPUs, and NPUs into a cohesive platform. The Mali and Immortalis GPUs, Arm's graphics lines, also support AI workloads, with Immortalis adding hardware ray tracing. This integration allows developers to optimize AI models across different compute units, maximizing efficiency. Arm AI's software tools, including the Arm Compute Library and CMSIS-NN for Cortex-M, further simplify deployment.
Market Position and Competition
Arm AI operates in a competitive landscape. In mobile AI, rivals include Qualcomm with its Hexagon DSP and Apple with its Neural Engine. In edge AI for IoT, competitors include Intel with Movidius and Google DeepMind's hardware partners. However, Arm's licensing model gives it a unique advantage: rather than selling chips, it licenses designs to hundreds of partners, including Samsung Electronics, TSMC, and Broadcom. This ecosystem approach has led to Arm-based processors being used in virtually all modern smartphones, as of 2025.
In the server and data center space, Arm AI faces competition from AMD and Intel, but its focus remains on edge. Arm's main CPU competitors in servers include IBM, Intel, and AMD, but Arm has no competition in mobile CPUs, though chip vendors compete within that space. For GPUs, Arm competes with Imagination Technologies, Qualcomm, and increasingly Nvidia, AMD, Samsung, and Intel. Despite this, Arm's NPUs are often integrated with GPUs from other vendors, as seen in Qualcomm's Snapdragon platforms.
Software and Developer Ecosystem
Arm AI provides a comprehensive software stack to support AI development. The Arm NN SDK, first released in 2017, allows developers to run neural networks on Arm hardware, supporting frameworks like TensorFlow, Caffe, and ONNX. In 2019, Arm introduced the Arm ML Embedded Evaluator (Arm ML Eval), a tool for benchmarking AI performance on embedded devices. The CMSIS-NN library, part of the Cortex Microcontroller Software Interface Standard, optimizes neural network kernels for Cortex-M processors.
Arm also collaborates with cloud providers to enable edge-to-cloud AI. Through partnerships with Amazon Web Services, Microsoft Azure, and Google Cloud, Arm AI designs are used in cloud instances for training, while edge devices handle inference. This hybrid approach is common in Generative AI applications, where models are trained in the cloud and deployed on-device. Arm's support for Model Pruning and Quantization helps reduce model size, making deployment feasible on memory-limited devices.
Recent Developments and Future Directions
In 2023, Arm went public on the Nasdaq, raising $4.87 billion at a $54.5 billion valuation, with SoftBank Group retaining about 90% ownership. This IPO provided capital for Arm AI to expand its product roadmap. In 2024, Arm announced the Ethos-U85, which supports transformer models, and introduced the Arm AI Accelerator for data centers, targeting inference at scale. These moves signal a push into more powerful edge AI, including on-device Large language model inference.
Arm AI is also exploring partnerships with AI research labs. In 2025, it collaborated with MIT CSAIL and Stanford AI Lab on energy-efficient AI algorithms, aiming to reduce the carbon footprint of AI. The division is investing in Residual Network (ResNet) and U-Net optimizations for computer vision, and in Sequence-to-Sequence (Seq2Seq) models for natural language processing. As AI models become more efficient, Arm AI aims to enable their deployment in billions of devices, from smart home sensors to autonomous vehicles.
Challenges and Controversies
Arm AI faces challenges, including the 2020-2022 attempted acquisition by Nvidia, which was cancelled due to regulatory pressure from the UK, EU, and US. This uncertainty affected Arm's strategic direction, but the company remained independent. Additionally, Arm's China subsidiary, Arm China, has been a source of dispute, with leadership conflicts reported in 2020. These issues have not directly impacted Arm AI's product development but highlight governance complexities.
Another challenge is the competitive pressure from open-source AI hardware initiatives, such as RISC-V, which offer free alternatives. However, Arm's mature ecosystem and software support provide a strong moat. As of 2026, Arm AI continues to innovate, with reports of a potential acquisition of Cerebras Systems, a wafer-scale chip maker, though the approach was rebuffed. This indicates Arm's ambition to expand into high-performance AI compute, potentially bridging edge and data center markets.
Conclusion
Arm AI represents a strategic focus within Arm Holdings on enabling AI at the edge. Through its Ethos NPU family and integration with Arm CPUs, it provides efficient, scalable solutions for a wide range of devices. The division's success is tied to Arm's licensing model, which fosters innovation across the semiconductor industry. As AI becomes ubiquitous, Arm AI is well-positioned to play a central role in bringing intelligence to every connected device, from wearables to industrial sensors, while addressing the growing demand for on-device processing in a privacy-conscious world.