SSML Dialogue Reliability

This paired benchmark measures the same ten-turn English dialogue through standard SSML and SSML-H. Standard SSML uses one generated voice with explicit 120 ms turn boundaries; SSML-H uses the dynamic Host and Guest definitions shown in the product example. Each mode receives one unmeasured warmup followed by five fixed-seed rounds by default.

All WAV files are generated and staged before the separate Qwen3-ASR phase. A measured call passes only when transcript similarity is at least 95%, all ten turn-ending phrases occur in order, and the transcript reaches within a single structure. This directly catches progressive truncation while retaining timing, real-time factor, audio duration, transcript, and audio hash evidence for future comparisons.

ASR is a consistent semantic judge, not a subjective voice-quality score. Failed rows require listening to the retained or independently reproduced WAV before assigning the failure to synthesis rather than recognition.

Run a full result with task benchmark-ssml. Validate wiring without changing history with task benchmark-ssml-smoke. The TTS and ASR URLs can be overridden through Task variables or direct runner arguments.

Run Mode Rounds Passed Pass rate Tail complete Turn endings Mean similarity Minimum similarity Mean TTS seconds Median RTF ASR model
08.10.2026 14:57:14 - 1.0-snapshot Standard SSML 5 5 100.00% 100.00% 100.00% 99.92% 99.66% 14.055 0.2965 Qwen/Qwen3-ASR-0.6B-hf
08.10.2026 14:57:14 - 1.0-snapshot SSML-H 5 5 100.00% 100.00% 100.00% 99.83% 99.13% 16.717 0.3505 Qwen/Qwen3-ASR-0.6B-hf