Overview
Chatterbox-Flash represents a zero-shot text-to-speech (TTS) model developed by fine-tuning a pretrained autoregressive TTS decoder. This fine-tuning transforms the decoder into a block-diffusion decoder, facilitating the parallel generation of tokens within each block while maintaining block-by-block streaming capabilities.
Research Context
The development of Chatterbox-Flash addresses specific challenges encountered when applying mainstream block-diffusion decoding to discrete speech tokens. A naive transfer of these techniques often results in degraded synthesis quality. This degradation is attributed to a long-tail token distribution, which tends to bias parallel position selection towards a limited set of high-frequency tokens. The research aimed to mitigate this issue without necessitating architectural modifications to the underlying model.
Approach
The core approach involved fine-tuning an existing autoregressive TTS decoder into a block-diffusion decoder. To address the quality degradation observed with discrete speech tokens, Chatterbox-Flash introduces two distinct inference-time techniques:
- Prior-calibrated scoring: This technique involves subtracting the block-level marginal token distribution from the scoring mechanism. Its purpose is to counteract the bias introduced by long-tail token distributions.
- Early-decoding schedule: This adaptive schedule is designed to terminate iteration based on calibrated confidence. This allows for efficient processing by stopping the decoding process once a sufficient level of confidence is achieved.
These techniques were implemented to improve the quality of synthesis under block-diffusion decoding for discrete speech tokens without altering the model's architecture.
Findings
On standard zero-shot TTS benchmarks, Chatterbox-Flash demonstrated several key findings:
- It achieved high-fidelity synthesis.
- The synthesis quality was comparable to strong autoregressive and non-autoregressive baselines.
- The model supports streaming inference.
- Its time-to-first-packet performance was on par with existing streaming autoregressive systems.
- It exhibited a substantially lower real-time factor compared to other methods.
Why This Matters
The findings indicate that Chatterbox-Flash offers a method for achieving high-quality zero-shot text-to-speech synthesis while enabling efficient streaming capabilities. The described inference-time techniques address specific challenges related to token distribution in block-diffusion decoding, which could contribute to more robust and performant TTS systems for various applications requiring real-time audio generation. The comparable performance to strong baselines suggests its viability as an alternative or enhancement in relevant contexts.