This page contains AI-generated content
Paper Demo Qwen3-TTS • Post-Training Study
Expressive Zero-Shot Voice Cloning

Rethinking Post-Training for Expressive Zero-Shot Text-to-Speech

Zero-shot voice cloning typically emphasizes preserving the timbre and prosodic characteristics of a reference speaker. However, in real-world use, speech should adapt its delivery to the target text rather than remain tied to a fixed speaking style. In this work, we formulate this problem as text-adaptive prosody synthesis for zero-shot voice cloning and study how post-training can endow a cloning model with this capability.

Emotion 18.5 → 81.2 Emergent emotion score
Paralinguistics 38.4 → 73.9 Emergent prosody score
Pipeline 3 stages SFT/DPO → GSPO → OPSD

Framework overview

Post-training
Stage 1

Direct Preference Optimization

Creates a cold start by aligning the model with text-appropriate expressive speech, improving prosodic controllability before more complex optimization.

Stage 2

Group Sequence Policy Optimization

Jointly optimizes expressiveness, acoustic quality, intelligibility, and speaker similarity, stabilizing the end-to-end training signal.

Stage 3

On-Policy Self-Distillation

Transfers expressive behavior from strong-reference conditions to weak-reference conditions, enabling self-improvement without external labels.

Sample comparison

Reference + synthesis

Main results

Qwen3-TTS
Model / Stage Emotion (All) Paraling. (All) WER (EN) SIM (EN) UTMOS (EN)
Base Model 18.5 38.4 1.23 0.72 3.35
SFT (Stage 1 Exp 0) 59.5 53.9 1.66 0.60 3.47
DPO (Stage 1 Exp 1) 30.6 43.3 2.17 0.66 3.27
DPO (Stage 1 Exp 2) 66.8 57.6 1.65 0.55 3.09
GSPO (Stage 2) 69.6 66.3 1.51 0.62 3.32
OPSD (Stage 3) 81.2 73.9 1.55 0.59 3.36
Emotion 18.5 → 81.2, a +62.7 absolute gain, indicating clear emergence of expressive delivery.
Paralinguistics 38.4 → 73.9, showing that text-adaptive prosody is learned progressively through post-training.
Takeaway Each stage successively improves text-to-speech expressivity while keeping acoustic stability and speaker fidelity in check.

Please note that the reference audio samples on this page are sourced from the open-source dataset seed-tts-eval, which is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License.

All other audio samples presented are AI-generated synthetic content produced by the post-trained models in this study. Any organization or individual using these AI-generated synthetic audio samples shall comply with applicable laws and regulations, fulfill their respective obligations when further generating or disseminating such content, and shall not use them for impersonation, fraud, or any other illegal or infringing purposes.