# Tacotron 2

Tacotron 2 is a neural text-to-speech system developed by Google AI in 2018 that converts text to mel-spectrograms and uses a neural vocoder to generate speech waveforms. It demonstrated highly natural speech synthesis but required tens of hours of training data for acceptable quality.

Tacotron 2 is a [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) based [text-to-speech](https://www.wikiprompt.org/wiki/artificial-intelligence) system developed by Google AI and released in 2018. It marked a significant advancement in neural speech synthesis by demonstrating that an end-to-end [neural-network](https://www.wikiprompt.org/wiki/neural-network) architecture could produce highly natural-sounding speech from written text. The system consists of two main components: a recurrent sequence-to-sequence feature prediction network that generates mel-spectrograms from input text, and a modified WaveNet vocoder that converts those spectrograms into raw audio waveforms. This two-stage approach allowed Tacotron 2 to achieve near-human naturalness in its output, though it required substantial training data - typically tens of hours of recorded speech - to reach acceptable quality.

The development of Tacotron 2 built upon earlier work in deep learning speech synthesis. In September 2016, DeepMind released WaveNet, which first demonstrated that deep learning models could model raw waveforms and generate speech from acoustic features like spectrograms or mel-spectrograms. WaveNet was initially computationally expensive and slow, but a year later DeepMind unveiled Parallel WaveNet, a production model approximately 1,000 times faster than the original. Tacotron 2 integrated these advances with an attention-based sequence-to-sequence model, allowing the system to align input text with output speech features without requiring explicit alignment information.

## Architecture

Tacotron 2's architecture is an autoencoder with attention mechanisms that processes input text character by character. The encoder converts the character sequence into a hidden representation, which an attention module then aligns with frames of the output mel-spectrogram. The decoder generates mel-spectrogram frames autoregressively, predicting each frame based on previously generated frames and the attended encoder states. The loss function for training typically uses L1 (mean absolute error) or L2 (mean square error) losses, which impose constraints that output acoustic feature distributions must be Gaussian or Laplacian. In practice, the loss function is designed to place more penalty on the human voice band, approximately 300 to 4000 Hz, using a weighted combination of losses for the human band and other frequencies.

The acoustic features used by Tacotron 2 are mel-spectrograms, which capture the time-frequency relation of the speech signal. These features are sufficient to generate intelligible outputs, unlike Mel-frequency cepstral coefficients used in speech recognition, which reduce too much information for synthesis purposes. The final waveform is produced by the WaveNet vocoder, which factorizes the joint probability of a waveform into a product of conditional probabilities, enabling high-quality audio generation.

## Training Data Requirements

One of the key findings from Tacotron 2's development was its sensitivity to training data quantity. When trained on tens of hours of audio, the system achieved highly natural speech synthesis. However, with smaller datasets, output quality degraded noticeably. With approximately 2 hours of speech, Tacotron 2 maintained intelligible speech but with reduced naturalness. With just 24 minutes of training data, the system failed to produce intelligible speech altogether. This data requirement stood in contrast to later systems; in March 2020, the free text-to-speech website 15.ai launched, and its creator claimed that 15 seconds of training data was sufficient to clone a person's voice. This claim was later corroborated by OpenAI in 2024, representing a significant reduction from Tacotron 2's requirements.

## Influence and Successors

Tacotron 2's limitations in speed and data efficiency spurred subsequent research. In 2019, Microsoft Research introduced FastSpeech, which addressed the speed limitations of autoregressive models like Tacotron 2. FastSpeech used a non-autoregressive architecture with a feedforward transformer network and length regulation, enabling parallel sequence generation and one-shot prediction of the full mel-spectrogram sequence. This significantly reduced inference time while maintaining audio quality. The same year saw the release of HiFi-GAN, a generative adversarial network-based vocoder that improved waveform generation efficiency while producing high-fidelity speech. In 2020, Glow-TTS introduced a flow-based approach allowing fast inference and voice style transfer.

Tacotron 2 also influenced research in zero-shot speaker adaptation. In June 2018, Google proposed using pre-trained speaker verification models as speaker encoders to extract speaker embeddings. These encoders became part of neural text-to-speech models, enabling a single model to generate speech with various speaker styles and characteristics without retraining for each speaker.

## Legacy

Tacotron 2 is widely regarded as a landmark model in the history of neural text-to-speech. It demonstrated that end-to-end deep learning approaches could rival traditional concatenative and parametric synthesis methods in naturalness. Its two-stage design - acoustic feature generation followed by neural vocoding - became a common paradigm in subsequent systems. While later models improved speed and data efficiency, Tacotron 2's contributions to attention-based sequence-to-sequence modeling for speech and its integration with neural vocoders remain influential in the field of [machine learning](https://www.wikiprompt.org/wiki/machine-learning) and [generative AI](https://www.wikiprompt.org/wiki/generative-ai).

---
Source: https://www.wikiprompt.org/wiki/tacotron-2
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-07T21:36:21.997608+00:00