PyTorch is an open-source deep learning framework that provides tools for building, training, and deploying neural networks, known for its Python-first design and dynamic, define-by-run computation graph. Originally developed by Facebook AI Research, later Meta AI, it was first released in September 2016 and became, by the early-to-mid 2020s, the dominant framework used in AI research.
History
PyTorch grew out of Torch, an earlier scientific computing framework built on the Lua programming language, and was rewritten with a Python interface to fit more naturally into the Python-centric data science ecosystem researchers already used. Its early design decisions, especially its dynamic computation graph, which builds the network's structure on the fly as code executes rather than requiring it to be defined and compiled in advance, made debugging and experimenting with novel architectures substantially easier than with the static-graph approach used by early TensorFlow. This developer-friendliness drove rapid adoption in academic research: by the early 2020s, the large majority of papers at major machine learning conferences used PyTorch. In September 2022, Meta transferred PyTorch's governance to the newly formed PyTorch Foundation under the Linux Foundation, reflecting its status as critical open-source infrastructure used across the industry rather than a single company's internal tool.
Design and ecosystem
PyTorch represents data as tensors, multidimensional arrays similar to those in NumPy, and provides automatic differentiation to compute gradients for gradient descent-based optimization automatically. It runs efficiently on both CPUs and, via CUDA, Nvidia GPUs, and has been extended with tools for distributed training across many GPUs, model compilation for performance, and mobile and edge deployment. A large ecosystem of libraries built on top of PyTorch, including Hugging Face's Transformers library, has made it the default substrate for training and fine-tuning transformer-based models, including most major open-weight large language models released in the 2020s.
Comparison to TensorFlow and significance
PyTorch's chief historical rival is Google's TensorFlow, released a year earlier in 2015; the two frameworks' competition shaped much of deep learning tooling through the late 2010s, with TensorFlow initially favored for production deployment and PyTorch for research flexibility. TensorFlow later adopted eager execution to close this gap, but PyTorch had already won a decisive share of research mindshare and, as the large language model era made research-to-production timelines shorter, much of production deployment as well. By the mid-2020s, PyTorch was the framework underlying most widely used foundation models and was frequently cited, alongside CUDA and Nvidia GPUs, as one of the pieces of infrastructure the modern AI industry depends on most heavily.