Whisper is an automatic Speech recognition system released by OpenAI in September 2022 as an open-source model, with both code and trained weights made publicly available. Unlike most of OpenAI's later releases, Whisper was published with a detailed technical paper describing its training data and methodology, making it one of the more transparent releases from a major AI lab during this period.
Training and architecture
Whisper is a Transformer (architecture)-based encoder-decoder model trained in a largely supervised, weakly labeled fashion on 680,000 hours of audio collected from the web, paired with corresponding transcripts. Roughly a third of that audio was non-English, spanning around 100 languages, and a portion of the training data consisted of translation pairs, giving the model built-in speech-to-text translation capability alongside transcription. This scale and diversity of weakly supervised data was a deliberate contrast to earlier speech recognition systems trained on smaller, more curated academic datasets, echoing broader findings in the field about scaling and data diversity improving robustness. The model was released in several sizes, from a lightweight "tiny" version suitable for on-device use to a large model offering the highest accuracy, an approach later followed by v2 and v3 updates that improved performance in several languages.
Adoption and impact
Because Whisper's weights were released under a permissive open license, it was rapidly integrated into a wide range of third-party products and open-source projects, including meeting transcription tools, subtitle generators, voice assistants, and accessibility software. Its robustness to accents, background noise, and technical vocabulary, relative to prior commercial ASR systems, made it a common default choice for developers building on Natural language processing pipelines that needed a speech-to-text front end. Whisper also became a component in larger multimodal and voice-interaction systems, feeding transcribed text into downstream large language models for voice assistants and call-center automation, and its transcripts have been used as a source of training data for other audio and speech projects.
The model's release contributed to a broader trend of foundation-model style open-weights releases from major labs even as those same labs kept their flagship generative models closed, a pattern also seen with CLIP a year earlier. Because it lowered the cost of building transcription and translation features, Whisper was frequently cited as a factor accelerating experimentation with voice-driven AI applications, including early integrations feeding into Text-to-speech and Voice cloning pipelines for dubbing and localization products, and it remains widely used as a benchmark and backbone in academic and industry speech research as of 2025.
Limitations
Whisper is not without weaknesses: it can hallucinate text that was never spoken, particularly during silences or non-speech audio, a failure mode related to but distinct from the broader Hallucination (AI) problem seen in generative language models. Accuracy also varies substantially across languages, with lower-resource languages showing higher error rates than English and other well-represented languages in its training data, and researchers have documented cases of the model inserting fabricated phrases into medical and other high-stakes transcripts, prompting caution about unsupervised use in sensitive contexts.