f5 tts: Fluent, Faithful Speech with Flow Matching

Thanks to all the contributors !

News

• 2025/03/12: A newer F5-TTS v1 base model introduced training and inference updates. Explore inference.
• 2024/10/08: F5-TTS and E2 TTS base-model resources were published for the research community. See installation.

Installation

Create a separate environment if needed

# Create a conda environment with Python 3.10 or newer
conda create -n f5-tts python=3.11
conda activate f5-tts

# Install FFmpeg if needed
conda install ffmpeg

Install PyTorch with matched device

# Select a PyTorch build matched to your CUDA version
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu128

# Select a PyTorch build matched to your ROCm version
pip install torch torchaudio --index-url https://download.pytorch.org/whl/rocm7.2

# Check the current PyTorch XPU installation guidance
pip install torch torchaudio --index-url https://download.pytorch.org/whl/test/xpu

pip install torch torchaudio

Then you can choose one from below:

1. As a pip package (if just for inference)

pip install f5-tts

2. Local editable (if also do training, finetuning)

git clone https://github.com/SWivid/F5-TTS.git
cd F5-TTS
# Optional when using BigVGAN as a vocoder:
# git submodule update --init --recursive
pip install -e .

Docker usage also available

# Build from the Dockerfile
docker build -t f5tts:v1 .

# Run the published container with a GPU
docker container run --rm -it --gpus=all -p 7860:7860 ghcr.io/swivid/f5-tts:main

# Launch its web interface
docker container run --rm -it --gpus=all -p 7860:7860 ghcr.io/swivid/f5-tts:main f5-tts_infer-gradio --host 0.0.0.0

Runtime

The project also documents deployment with Triton and TensorRT-LLM.

Benchmark Results

The published benchmark describes decoding on a single L20 GPU across 26 prompt-audio and target-text pairs at 16 NFE.

See the project's runtime documentation for benchmark details and setup conditions.

Inference

• Read the inference guidance before evaluating model performance.
• Search existing project discussions when troubleshooting a specific issue.

1. Gradio App

Documented features include:

• Basic TTS with chunk inference
• Multi-style and multi-speaker generation
• Voice chat using Qwen2.5-3B-Instruct
• Custom inference with additional language support

# Launch the Gradio web interface
f5-tts_infer-gradio

# Specify a port and host
f5-tts_infer-gradio --port 7860 --host 0.0.0.0

# Request a share link
f5-tts_infer-gradio --share

services:
  f5-tts:
    image: ghcr.io/swivid/f5-tts:main
    ports:
      - "7860:7860"
    environment:
      GRADIO_SERVER_PORT: 7860

2. CLI Inference

# Run with a reference recording and a new text prompt
f5-tts_infer-cli --model F5TTS_v1_Base \
--ref_audio "provide_prompt_wav_path_here.wav" \
--ref_text "The words spoken in the reference audio." \
--gen_text "The words to generate."

# Run with the project's default settings
f5-tts_infer-cli

# Or provide your own TOML settings
f5-tts_infer-cli -c custom.toml

# A multi-voice example
f5-tts_infer-cli -c src/f5_tts/infer/examples/multi/story.toml

Training

1. With Hugging Face Accelerate

Refer to the project's training and finetuning guidance for recommended practices.

2. With Gradio App

# Start the finetuning web interface
f5-tts_finetune-gradio

Consult the training guidance for configuration and dataset instructions.

Evaluation

Development

Pre-commit can run the project's formatters and quality checks:

pip install pre-commit
pre-commit install

Before submitting changes, run the checks across the files:

pre-commit run --all-files

Some model components have linting exceptions for tensor notation.

Acknowledgements

• E2-TTS for its related flow-matching research
• Emilia, WenetSpeech4TTS, LibriTTS and LJSpeech datasets
• Lucidrains and collaborators for early CFM structures and discussion
• Stable Diffusion 3 and Hugging Face Diffusers for transformer code references
• Torchdiffeq, Vocos and BigVGAN for solver and vocoder work
• FunASR, faster-whisper, UniSpeech and SpeechMOS for evaluation tools
• CTC forced aligner for speech-edit evaluation
• Community members who built speech demos
• The MLX implementation and its maintainers
• The ONNX Runtime implementation and its maintainers
• Contributors to Triton and TensorRT-LLM support

Citation

If the research and codebase are useful to you, cite the original paper:

@article{chen-etal-2024-f5tts,
  title={F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching},
  author={Yushen Chen and Zhikang Niu and Ziyang Ma and Keqi Deng
          and Chunhui Wang and Jian Zhao and Kai Yu and Xie Chen},
  journal={arXiv preprint arXiv:2410.06885},
  year={2024}
}

License

The original project's code is released under the MIT License. Its pretrained models have separate CC-BY-NC licensing associated with their training data. Check the original project terms before using either in your own work.