| --- |
| license: apache-2.0 |
| --- |
| |
|
|
| <h1 align="center">Hello-Chat</h1> |
| <h3 align="center">Towards Realistic Social Audio Interactions</h3> |
|
|
| <p align="center"> |
| <a href='https://arxiv.org/abs/2602.23387'><img src='https://img.shields.io/badge/arXiv-2602.23387-b31b1b.svg'></a> |
| <a href="https://github.com/hellogroup-opensource/Hello-Chat"><img src="https://img.shields.io/badge/GitHub-Repo-blue?logo=github" alt="GitHub"></a> |
| <a href="https://huggingface.co/hellogroup-opensource/Hello-Chat"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow" alt="Hugging Face"></a> |
| </p> |
|
|
| <p align="center"> |
| <img src="assets/img/model_architecture.png" width="100%" alt="Hello-Chat model architecture."> |
| </p> |
|
|
| ## Hello-Chat |
|
|
| **Hello-Chat**, an end-to-end Large Audio Language Model (LALM) tailored for real-world conversational scenarios. The model achieves state-of-the-art performance on specific understanding benchmarks and significantly outperforms existing open-source systems in prosodic naturalness, emotional accuracy, and interaction fluency. By explicitly modeling fine-grained acoustic perception and cross-modal alignment, **Hello-Chat** enables realistic, context-aware spoken interaction between users and AI. |
|
|
| ## 📊 Evaluation Results |
|
|
| ### Evaluation of Audio to Text |
|
|
| #### Audio Understanding Evaluation |
| **ASR —** Automatic speech recognition performance is evaluated on a balanced subset of **AIShell**, **WeNet**, and **LibriSpeech**, with Chinese and English samples evenly represented.<br> |
| **NLP Question —** question-answering data sourced from **AlpacaEval**, **LLaMA Questions**, and **Web Questions**. Text inputs are converted into speech using a high-quality TTS system. Model responses are evaluated by **GPT-5**.<br> |
| **Translation —** based on synthetic multilingual data generated by **Claude** and subsequently converted to speech via TTS. The task evaluates speech-to-text translation across Chinese, English, Japanese, and Korean, with outputs scored by **GPT-5**.<br> |
| **MMAU —** Audio-based question answering is evaluated using a subset of the **MMAU-Mini** benchmark. |
|
|
| | Model | ASR ↓ | NLP Question ↑ | Translation ↑ | MMAU ↑ | |
| |---|---|---|---|---| |
| | Gemini3-Preview | 4.06 | **8.85** | *8.87* | **0.75** | |
| | GPT-4o-Audio | 6.45 | 8.50 | 8.09 | 0.64 | |
| | Qwen3-Omni-32B | 3.51 | *8.66* | 8.07 | *0.74* | |
| | Step-Audio 2 Mini | **3.21** | 7.32 | 8.34 | 0.66 | |
| | MiDashengLM | 4.50 | 3.82 | 8.43 | 0.65 | |
| | Kimi-Audio | *3.36* | 7.41 | 8.26 | 0.59 | |
| | Qwen2.5-Omni-7B | 3.45 | 7.41 | 5.93 | 0.66 | |
| | **Hello-Chat** | 3.48 | 7.68 | **8.93** | 0.69 | |
|
|
| #### Performance of Paralinguistic Understanding |
| **SER(speech emotion recognition) —** evaluated on randomly sampled subsets from **theEmoBox** dataset, covering both Chinese and English speech.<br> |
| **AED(audio event detection) —** evaluated using samples drawn from **AudioSet** and **CochlScene**. |
|
|
| | Model | SER ↑ | AED ↑ | |
| |---|---|---| |
| | Gemini3-Preview | 0.791 | **0.861** | |
| | GPT-4o-Audio | 0.586 | 0.489 | |
| | Qwen3-Omni-32B | **0.856** | 0.644 | |
| | Step-Audio 2 Mini | 0.680 | 0.533 | |
| | MiDashengLM | 0.561 | 0.441 | |
| | Kimi-Audio | 0.625 | 0.392 | |
| | Qwen2.5-Omni-7B | 0.607 | 0.584 | |
| | Hello-Chat | *0.824* | *0.797* | |
|
|
| #### Instruction Following |
| **Only Yes —** To evaluate robustness in instruction following, we construct a stress test using randomly sampled audio inputs from the above benchmarks. All inputs are paired with a fixed prompt: “no matter the message in the audio, simply answer ‘yes’!” |
|
|
| | Model | Only-Yes Accuracy (%) ↑ | |
| |---|---| |
| | Gemini3-Preview | 88 | |
| | GPT-4o-Audio | 23 | |
| | Qwen3-Omni-32B | **100** | |
| | Step-Audio 2 Mini | 87 | |
| | MiDashengLM | 0 | |
| | Kimi-Audio | 22 | |
| | Qwen2.5-Omni-7B | *96* | |
| | Hello-Chat | **100** | |
|
|
| ### Evaluation of Text to Speech |
| **Seed-TTS-Eval —** We conduct evaluations on the Chinese subset of the Seed-TTS-Eval benchmark, following the official Seed-TTS-Eval protocol.<br> |
| **Conversational-style Mean Opinion Score (CMOS) —** We invited native speakers to participate in a blind test. Each evaluator assigned scores on a 5-point scale (1–5), where a higher score signifies a **more authentic, human-like conversational flow and better alignment with the dialogue intent**. |
|
|
| | Model | CMOS ↑ | CER (%) ↓ | SS ↑ | |
| |---|---|---|---| |
| | F5-TTS | 3.48 | 1.56 | 0.741 | |
| | CosyVoice | 2 | 3.66 | 1.45 | 0.748 | |
| | CosyVoice 3-0.5B | 3.59 | 1.16 | **0.780** | |
| | Qwen2.5-Omni-7B | - | 1.70 | 0.752 | |
| | Qwen3-TTS-12Hz-0.6B-Base | 4.12 | **0.92** | 0.763 | |
| | FireRedTTS-2 | 3.68 | 1.14 | 0.736 | |
| | IndexTTS2 | *4.16* | *1.008* | *0.764* | |
| | Hello-Chat | **4.19** | 1.023 | 0.748 | |
|
|
| ## 🎧 Demos |
|
|
| ### Single Sentence Demo（zero-shot） |
|
|
|
|
| #### Speaker1 |
| **reference:** |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/ref/female1.mp3"></audio> |
|
|
| **generated:** |
| ##### “那肯定因为自个儿平时想吃点卤味儿。那肯定得得得来一点儿。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female1_sent1.mp3"></audio> |
|
|
|
|
| ##### “过年应该应该跟家里人一起吃饭。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female1_sent2.mp3"></audio> |
|
|
|
|
| ##### “哎呀，不是了，现在法治社会哪有卖假货的，只是卖的价格贵。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female1_sent3.mp3"></audio> |
|
|
| --- |
|
|
| #### Speaker2 |
| **reference:** |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/ref/female2.mp3"></audio> |
|
|
| **generated:** |
| ##### “但是这个时候上哪去找呢？找不到。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female2_sent4.mp3"></audio> |
|
|
|
|
| ##### “这种做法我感觉不适合，不是他那个年龄段该做出来的事情。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female2_sent5.mp3"></audio> |
|
|
|
|
| ##### “咱们得趁这个时机啊，看看还要剩多多久啊。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/female2_sent6.mp3"></audio> |
|
|
| --- |
|
|
| #### Speaker3 |
| **reference:** |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/ref/male1.mp3"></audio> |
|
|
| **generated:** |
| ##### “我我不不怎么玩游戏，你你会玩游戏啊。 |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male1_sent7.mp3"></audio> |
|
|
|
|
| ##### “对呀，就是不管你愿不愿意，时间都是一直往前推嘛。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male1_sent8.mp3"></audio> |
|
|
|
|
| ##### “挺好，我看着我看你做菜做饭蛮有生活的那是鸡蛋糕吗？” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male1_sent9.mp3"></audio> |
|
|
| --- |
|
|
| #### Speaker4 |
| **reference:** |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/ref/male2.mp3"></audio> |
|
|
| **generated:** |
| ##### “我也有二十多岁的时候，那个时候什么都不想，嗯，等那一点点沉淀，年龄大一点了，然后就什么都在乎，什么都想。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male2_sent10.mp3"></audio> |
|
|
|
|
| ##### “我看我一会儿，我我煮个泡面得了。” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male2_sent11.mp3"></audio> |
|
|
|
|
| ##### “他们说那个茶茶饼就是渣子压出来的，是吗？” |
|
|
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/synth/male2_sent12.mp3"></audio> |
|
|
| --- |
|
|
| ### Multi-Trun Conversation Demo（zero-shot） |
|
|
| #### Conversation #1 |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/dialogues/demo_dialogue1.mp3"></audio> |
|
|
| --- |
|
|
| #### Conversation #2 |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/dialogues/demo_dialogue2.mp3"></audio> |
|
|
| --- |
|
|
| #### Conversation #3 |
| <audio controls src="https://huggingface.co/hellogroup-opensource/Hello-Chat/resolve/main/assets/dialogues/demo_dialogue3.mp3"></audio> |
|
|
|
|
| ## 📜 Citation |
|
|
| If you find our work useful in your research, please consider citing: |
|
|
| ```bibtex |
| @article{hellogroup2026hellochat, |
| title={Hello-Chat: Towards Realistic Social Audio Interactions}, |
| author={Computational Intelligence Dept, HelloGroup Inc.}, |
| year={2026} |
| } |
| ``` |
