IndexTTS2 Zero Shot Voice Cloning Beats Benchmarks for Accuracy and Emotion

IndexTTS2 addresses a persistent issue in modern speech synthesis: the difficulty of achieving precise speech timing. Instead of relying on duration estimates, it utilizes a token-counting method that fixes speech length while preserving the original speaker's rhythm and intonation. This ensures targeted durations without distorting the natural cadence of the prompt.

The model also decouples speaker identity from emotional tone, allowing for independent adjustments to both. In zero-shot mode, it reconstructs a target timbre and applies a specific emotional delivery—such as anger, sadness, or excitement—without the two attributes bleeding into each other. To maintain clarity in high-emotion clips, the system incorporates implicit GPT representations and a three-stage training regimen. A soft-prompt configuration, fine-tuned from Qwen3, simplifies emotional control; users can direct the output using basic prompts, and the model manages the complex mapping.

IndexTTS2 outperforms existing zero-shot TTS systems, demonstrating lower word error rates, higher speaker similarity, and superior emotional fidelity. The release includes a web-based demo and a Python API. Emotional inputs are flexible, accepting reference clips, standalone emotional samples, 8-dimensional vectors, or plain text.

Technical Architecture

The pipeline integrates several components: a 45E condition vector, a text-speech language model, a BigVGAN2 decoder, speaker vectors, text tokens, and both 5-dimensional and 8-dimensional latent acoustic tokens. Together, these elements synthesize the final waveform.

Key design features include:

1. Character-Pinyin Hybrid for Chinese: This modeling approach identifies and corrects mispronunciations in real-time.
2. Conformer + BigVGAN2 Backbone: This combination stabilizes the training process and improves voice similarity and audio fidelity.
3. Open Evaluation Suites: All evaluation sets—including polysyllabic word tests and subjective/objective metrics—are published to allow for transparent replication and research.

Performance Metrics

Word Error Rate (Seed-Test)

Word Error Rate (Public Benchmarks)

Speaker Similarity

Zero-Shot Voice Cloning (MOS Scores)

Quick Start: IndexTTS2 Setup

Environment

Clone the repository and create a new Conda environment:

git clone https://github.com/index-tts/index-tts.git
conda create -n index-tts python=3.10
conda activate index-tts
apt-get install ffmpeg
# or: conda install -c conda-forge ffmpeg

Install PyTorch (example for CUDA 11.8):

pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu118

Windows users encountering pynini build errors should install it via Conda:

conda install -c conda-forge pynini==2.1.6
pip install WeTextProcessing --no-deps

Install IndexTTS as an editable package:

cd index-tts
pip install -e .

Download Models

Use the huggingface-cli:

huggingface-cli download IndexTeam/IndexTTS-1.5 \
  config.yaml bigvgan_discriminator.pth bigvgan_generator.pth bpe.model dvae.pth gpt.pth unigram_12000.vocab \
  --local-dir checkpoints

For faster downloads in China, use the mirror endpoint:

export HF_ENDPOINT="https://hf-mirror.com"

Inference

Place a reference audio file named input.wav in the test_data folder, then run:

python indextts/infer.py

Command-line usage:

indextts "Your text here." --voice reference_voice.wav --model_dir checkpoints --config checkpoints/config.yaml --output output.wav

Web Demo

pip install -e ".[webui]" --no-build-isolation
python webui.py

Access the interface at http://127.0.0.1:7860.

Python API Example

from indextts.infer import IndexTTS

tts = IndexTTS(model_dir="checkpoints", cfg_path="checkpoints/config.yaml")
voice = "reference_voice.wav"
text = "Your long-form text goes here."

tts.infer(voice, text, "output.wav")