Overview
This document details a self-designed system developed for Task 1 of the MLC-SLM 2026 Challenge, which focuses on multilingual two-speaker conversational speech. The system combines a modular speaker diarization front end with a modified Qwen3-ASR-1.7B recognizer adapted for the challenge's specific requirements. The primary objective of this system is to accurately transcribe two-speaker multilingual conversations.
Research Context
The system was developed within the framework of the MLC-SLM 2026 Challenge, specifically for Task 1, which addresses multilingual two-speaker conversational speech. The core components include a speaker diarization front end and an Automatic Speech Recognition (ASR) model, Qwen3-ASR-1.7B, which underwent specific adaptation procedures to perform within this challenge context. The adaptation strategies aimed to improve performance on the challenge's datasets.
Approach
The system is structured as a two-stage process, integrating speaker diarization followed by ASR. The speaker diarization front end incorporates several sequential operations:
- Voice activity detection.
- Subsegment generation from detected speech activity.
- CAMPPlus speaker embedding extraction.
- Two-speaker spectral clustering based on extracted embeddings.
- RTTM-based audio segmentation to define speaker-attributed segments.
Following diarization, the resulting speaker-attributed audio segments are categorized by language or region. These categorized segments are then processed by the adapted Qwen3-ASR-1.7B model for decoding.
The ASR adaptation process involved a three-phase strategy:
- Supervised Full Fine-Tuning: Initial fine-tuning was performed using the official training data provided for the challenge.
- LoRA Fine-Tuning with Synthetic Speech: This phase utilized synthetic speech generated by a three-pipeline Text-to-Speech (TTS)-based synthetic speech augmentation framework. Low-Rank Adaptation (LoRA) was applied during this stage.
- Reinforcement Learning Refinement: The final refinement stage employed Generative Reinforcement Policy Optimization (GRPO). Rewards for GRPO were based on Word Error Rate (WER) and Character Error Rate (CER), while penalties were applied for identified issues such as hallucination, repetition, and length deviation in the ASR output.
Findings
The system's performance was evaluated on both the official development and final evaluation sets of the MLC-SLM 2026 Challenge:
- On the official development set, the full system achieved an average tcpMER of 23.70. This represented a reduction of 6.83 absolute points in error rate when compared to the performance of the vanilla Qwen-ASR-1.7B baseline model.
- On the final evaluation set, the system achieved an average tcpMER of 17.97.
Ablation studies provided insights into the contribution of each adaptation step:
- Supervised fine-tuning was identified as providing the largest performance gain.
- Both LoRA adaptation using synthetic speech and the reinforcement learning phase further improved the system's robustness.