TTS Voice-Generation Benchmark
This benchmark measures KokoroTTS end to end through the public /tts/generate HTTP endpoint. examples/voices.js is the single source of benchmark voices and language-specific input text, so the benchmark automatically follows the examples page as voices are added or corrected.
An official run performs these steps:
- Load every voice and its localized introduction text from
examples/voices.js. - Generate one unmeasured warmup for the first voice in each language.
- Generate WAV audio five times for every voice, rotating through all voices once per round.
- Write raw results and overall, per-language, and per-voice aggregates to
results/latest.json. - Append the overall result to
BENCHMARKS.mdand detailed language/voice tables toDETAILS.md.
Start the application, then run:
task benchmark-smoke
task benchmark-tts BENCHMARK_COMMENT="baseline"
benchmark-smoke checks the runner with two voices and one measured generation each without changing benchmark history. benchmark-tts measures all 56 current voices five times each, following one warmup request per language.
Useful overrides:
task benchmark-tts BENCHMARK_REPETITIONS=2 BENCHMARK_LIMIT=9
task benchmark-tts GPU_DEVICE=1 APP_DEVICE=cuda:0 BENCHMARK_COMMENT="second host GPU"
Keep the same image, host GPU, device selection, voice source, and repetition count when comparing runs. These numbers measure generation performance; voice quality still requires listening tests.