AWARD    MERL Team Wins Real-TSE Challenge Track 2 on Offline Target Speaker Extraction

Date released: September 14, 2026

  •  AWARD    MERL Team Wins Real-TSE Challenge Track 2 on Offline Target Speaker Extraction
  • Date:

    July 6, 2026

  • Awarded to:

    Dominik Klement, Yoshiki Masuyama, Christoph Boeddeker, Kohei Saijo, Julius Richter, Gordon Wichern, and Jonathan Le Roux

  • Description:

    MERL's Speech & Audio team, led by MERL intern Dominik Klement, ranked 1st out of 11 teams in Track 2, "Offline Target Speaker Extraction," of the Real-TSE Challenge. The challenge focuses on target speaker extraction (TSE) from real-world conversational recordings in either English or Chinese, where the goal is to extract the speech of a target speaker in the presence of interfering speakers, background noise, and reverberation.

    While modern TSE systems have achieved strong performance on simulated speech mixtures, their performance can degrade considerably on real-world recordings due to the mismatch between simulated training data and actual conversational environments. The Real-TSE Challenge was designed to advance TSE under these realistic conditions, using real far-field conversational recordings for evaluation.

    Four-stage training pipeline for real-world target speaker extraction Fully-overlapped pre-training, followed by simulated conversation pre-training, simulated far-field mixtures fine-tuning, and real far-field mixtures fine-tuning. The first two stages use single-talker clean speech, noises, and room impulse responses. The third uses single-talker far-field speech; the fourth uses multi-talker far-field mixtures. 12Fully-overlappedPre-trainingSim. ConversationPre-trainingSimulatedFar-field MixturesFine-tuningSingle-talkerCleanSpeechNoises&RIRsSingle-talkerFar-fieldSpeech3RealFar-field MixturesFine-tuningMulti-talkerFar-fieldMixtures4

    The MERL team won Track 2 by focusing on training data and curriculum learning rather than introducing a new model architecture. Starting from a strong speech separation model, the team progressively trained the system on fully overlapping synthetic speech, simulated conversations, realistic far-field mixtures, and finally real conversational recordings. This approach reduced the token error rate (TER), measured at either the word (English) or character (Chinese) level, from 70% to 37% on the development set and achieved a final TER of 61.3% on the evaluation set, best among the 11 participating teams. The team also topped the leaderboard in terms of the aggregate ranking across the four measures evaluating intelligibility, target speaker presence rate, speaker similarity, and perceptual quality.

    The team also investigated the reliability of the challenge metrics and demonstrated that neural network-based speaker similarity and predicted speech-quality scores could be substantially improved without a corresponding improvement in perceptual quality. Because learned metrics can be susceptible to adversarial attacks or optimization that exploits weaknesses in the metric itself, these findings highlight both the importance of realistic training data for real-world TSE and the need for robust evaluation metrics when developing speech extraction systems.

    A paper summarizing the team's findings will be presented at the IEEE Spoken Language Technology (SLT) 2026 workshop, to be held in Palermo, Italy from December 13-16, 2026.

    REAL-TSE Challenge: Track 2 rankings — Offline Target Speaker Extraction
    RankTeamTER ↓F1 ↑SIM ↑P808 ↑Score ↓
    1 MERL 0.613 (1) 0.861 (2) 0.538 (3) 3.371 (2) 2.00
    2 YiJiaHe 0.639 (2) 0.871 (1) 0.565 (1) 3.128 (9) 3.25
    3 CARTSE 0.651 (3) 0.857 (4) 0.544 (2) 3.138 (8) 4.25
    4 WasedaM 0.675 (5) 0.858 (3) 0.480 (6) 3.232 (6) 5.00
    5 SonicAGI 0.680 (6) 0.851 (6) 0.471 (7) 3.258 (5) 6.00
    6 WAKA 0.670 (4) 0.847 (8) 0.471 (7) 3.150 (7) 6.50
    6 SHNU-TSE 0.731 (9) 0.840 (9) 0.507 (5) 3.362 (3) 6.50
    7 ChuEst 0.710 (7) 0.831 (11) 0.532 (4) 3.064 (10) 8.00
    8 pyannoteAI 0.728 (8) 0.855 (5) 0.464 (9) 2.904 (12) 8.50
    9 AGH-JHU 0.743 (10) 0.837 (10) 0.434 (11) 3.335 (4) 8.75
    10 WHU_IASP 0.757 (11) 0.850 (7) 0.465 (8) 2.961 (11) 9.25
    11 CUDA_OUT_OF_MEMORY 0.827 (12) 0.819 (13) 0.364 (13) 3.435 (1) 9.75
    12 BSRNN_EMB Baseline 0.829 (13) 0.829 (12) 0.417 (12) 2.875 (13) 12.50
    12 BSRNN_TFMAP Baseline 0.838 (14) 0.829 (12) 0.443 (10) 2.756 (14) 12.50

    ↓ Lower is better; ↑ higher is better. Parentheses show metric ranks. The score is the average of the four dense metric ranks; tied scores share a position. Best metric values are bold. P808 denotes DNSMOS-P808.

    Source: Official REAL-TSE Challenge rankings. BSRNN entries are organizer baselines.

  • MERL Contacts:
  • External Link:

    https://real-tse.github.io/challenge/

  • Research Areas:

    Artificial Intelligence, Machine Learning, Speech & Audio

    •  Klement, D., Masuyama, Y., Boeddeker, C., Saijo, K., Richter, J., Wichern, G., Le Roux, J., "Technical Report for MERL’s Real-TSE Challenge Submission," Tech. Rep. TR2026-112, Mitsubishi Electric Research Laboratories, July 2026.
      BibTeX TR2026-112 PDF
      • @techreport{Klement2026jul2,
      • author = {Klement, Dominik and Masuyama, Yoshiki and Boeddeker, Christoph and Saijo, Kohei and Richter, Julius and Wichern, Gordon and {Le Roux}, Jonathan},
      • title = {{Technical Report for MERL’s Real-TSE Challenge Submission}},
      • institution = {Real-TSE Challenge},
      • year = 2026,
      • month = jul,
      • url = {https://www.merl.com/publications/TR2026-112}
      • }