Decart is a technology company specializing in artificial intelligence infrastructure. The organization develops software and hardware solutions designed to accelerate the training and inference of large-scale machine learning models, with a particular focus on improving computational efficiency and reducing the cost associated with AI workloads. The company operates within the broader context of the generative AI boom, addressing the growing demand for specialized compute resources.
Decart's work intersects with several key areas of modern AI development, including the optimization of Transformer (architecture) architectures and the deployment of large language models. The company's approach often involves close collaboration with hardware providers to maximize the performance of neural networks on specific chips, a practice that has become increasingly critical as the scale of AI models continues to expand.
Founding and Early Development
Decart was founded in 2023 by Dean Leitersdorf and Moshe Shalom. The company emerged from the Berkeley AI Research ecosystem, with Leitersdorf having previously been involved in academic research. The founders identified a critical bottleneck in the AI industry: the immense computational cost of training and running state-of-the-art models. Their initial focus was on developing a platform that could optimize the execution of models on various hardware, including AMD GPUs, which were often seen as a more cost-effective alternative to the dominant offerings from Nvidia.
Within its first year, Decart secured significant venture capital funding. In 2024, the company announced a $32 million Series A funding round led by Benchmark, a prominent Silicon Valley venture capital firm. This investment signaled strong market confidence in Decart's technology and its potential to disrupt the AI infrastructure landscape, which was then dominated by a few large cloud providers.
Technology and Core Products
Decart's primary product is an inference engine and orchestration platform designed to run AI models with exceptional speed and efficiency. The company's software stack is built to be hardware-agnostic, allowing it to optimize performance across a range of accelerators, including AWS Trainium chips, Google Cloud TPUs, and Intel accelerators, in addition to AMD GPUs. This flexibility is a key differentiator, as it enables customers to avoid vendor lock-in and choose the most cost-effective hardware for their specific needs.
A notable achievement for Decart was the development of a system capable of running a U-Net-based image generation model, such as Stable Diffusion, at speeds significantly faster than typical implementations. The company reported achieving inference speeds that were several times faster than those of leading competitors, partly through advanced techniques like model pruning and batch normalization optimizations. This performance was demonstrated in a live demo that ran a text-to-image model on AMD hardware, showcasing the potential for real-time, interactive AI applications.
Market Position and Partnerships
Decart positions itself as a challenger to established AI cloud providers like Oracle Cloud and Microsoft Azure. Instead of building massive, centralized data centers, Decart's platform is designed to aggregate and utilize distributed compute resources, including potentially underutilized consumer-grade GPUs. This approach aims to create a more efficient and accessible compute market, lowering the barrier to entry for AI startups and researchers.
The company has formed strategic partnerships with hardware manufacturers to co-optimize their software with new chip designs. A notable collaboration is with AMD, where Decart has worked to ensure its inference engine fully leverages the capabilities of AMD's Instinct series of accelerators. This partnership is part of a broader industry trend where software companies work closely with chip designers to close the performance gap with Nvidia's CUDA ecosystem.
Impact and Future Directions
The rise of Decart reflects a broader shift in the AI industry towards efficiency and cost optimization. As the cost of training and running generative AI models has skyrocketed, there is increasing pressure to find ways to do more with less. Decart's focus on inference speed is particularly relevant, as inference costs often dominate the total expense of operating AI services at scale.
The company's technology has implications for the development of artificial intelligence applications that require real-time responses, such as interactive agents, gaming, and autonomous systems. By making high-performance inference more affordable, Decart aims to accelerate the deployment of AI across various sectors. As of 2025, the company continues to expand its engineering team and develop new features, with a stated goal of becoming a foundational layer for the next generation of AI applications.