⭐ Selected Contributions
OmniCodec: Low Frame Rate Universal Audio Codec with Semantic-Acoustic DisentanglementFirst author
- Introduces a low-frame-rate universal codec for speech, music, and general audio.
- Separates semantic and acoustic information with a hierarchical multi-codebook design.
Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization GenerationFirst author
- Uses large audio-language model feedback to build preference pairs for non-verbal vocalization generation.
- Combines rejection-sampling fine-tuning with Anchored Flow-DPO; achieves a 75.80 Track 2 score.
Bridging Data, Reasoning, and Alignment: A Unified Framework for Context-Aware Instruction-Following TTSFirst author
- Builds a data cleaning and distillation pipeline for context-aware, instruction-following speech synthesis.
- Introduces Context-Aware DPO to align dialogue history, reasoning, instructions, and generated speech.
M5-TTS: A Mini-Scale Multilingual and Multi-Dialect Text-to-Speech Language Model with Multi-Token PredictionCo-first author
- Presents a compact 50M-parameter TTS language model for multilingual and Chinese multi-dialect speech.
- Uses multi-token prediction across 16 codec codebooks and supports zero-shot voice cloning and streaming.
WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing EcosystemCo-first author
- Introduces an approximately 8,000-hour open Wu dialect corpus with multidimensional annotations.
- Pairs the corpus with a benchmark and open models across speech understanding and generation tasks.
DRAMA-Eval: A Benchmark for Context-Aware, Multi-Turn, Multi-Speaker Speech GenerationFirst author
- Provides a 12-hour bilingual benchmark built from human-verified scripts and reference audio.
- Evaluates acoustic fidelity, local instruction execution, and long-range dialogue coherence.
Rethinking Post-Training for Expressive Zero-Shot Text-to-SpeechFirst author
- Studies text-adaptive prosody for zero-shot voice cloning through a three-stage post-training framework.
- Combines DPO, GSPO, and context-conditioned on-policy self-distillation for expressive synthesis.