Hibiki-Zero

### TL;DR

Hibiki-Zero is an advanced model for simultaneous speech-to-speech translation that eliminates the need for word-level aligned data, simplifying the training pipeline and enabling seamless scaling to diverse languages. It achieves state-of-the-art performance in translation accuracy, latency, voice transfer, and naturalness across multiple tasks.

Key Insights & Metrics

Key Features

• Simultaneous speech-to-speech translation without word-level aligned data
• State-of-the-art performance in translation accuracy and latency
• Preserves speaker identity and speech naturalness
• Adaptable to new languages with less than 1000 hours of speech data
• Provides model weights, inference code, and a 45-hour multilingual speech benchmark for evaluation

→ Related Releases

Discussion