Speech recognition, also called automatic speech recognition (ASR), converts spoken audio into written text. It is the input-facing counterpart to Text-to-speech and a foundational component of Conversational AI, voice assistants, dictation software, and automatic captioning systems.
History
Early speech recognition systems in the 1970s and 1980s relied on hand-engineered acoustic features combined with hidden Markov models, statistical models well suited to representing the sequential, time-varying structure of speech, often paired with n-gram language models to constrain plausible word sequences. These systems required substantial domain-specific tuning and struggled with accents, background noise, and vocabulary outside their training data. The 2010s brought a shift toward Deep learning, first by replacing individual pipeline components with neural networks and later by moving to fully end-to-end neural architectures, notably Recurrent neural network-based sequence models and, after 2017, Transformer (architecture)-based systems that mapped raw or lightly processed audio directly to text without a separate acoustic and language model pipeline.
Modern systems
OpenAI's Whisper, released as open-source in 2022, became a widely adopted general-purpose ASR system, trained on a large and diverse multilingual dataset of audio paired with transcripts, and notable for strong performance across accents, background noise, and multiple languages without per-language fine-tuning. Whisper and comparable systems from other labs are commonly used as a component within larger Multimodal AI pipelines, voice assistants, and meeting-transcription products, and their outputs frequently feed into downstream Natural language processing tasks such as summarization or translation.
Techniques
Modern ASR systems typically use an encoder that processes audio into a sequence of learned representations and a decoder that generates text autoregressively, an architecture pattern closely related to Seq2seq models and machine translation systems. Training relies heavily on large paired audio-text datasets, and Self-supervised learning pretraining on unlabeled audio, learning useful representations before any transcripts are available, became an important technique for improving performance on languages with limited labeled data, paralleling similar approaches in natural language processing and Computer vision.
Applications and limitations
Speech recognition underlies live captioning, voice assistants such as those built into smartphones, dictation software, call-center analytics, and accessibility tools for deaf and hard-of-hearing users. Despite major gains in accuracy, ASR systems still perform unevenly across accents, dialects, and languages with less training data available, a documented source of Algorithmic bias in commercial systems. Background noise, overlapping speakers, and domain-specific vocabulary such as medical or legal terminology also remain sources of error, and systems are frequently fine-tuned or supplemented with custom vocabularies for specialized deployments. Evaluation typically centers on word error rate, though that single metric can obscure uneven performance across demographic groups and speaking conditions.