TR2021-103

Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition

    •  Higuchi, Y., Moritz, N., Le Roux, J., Hori, T., "Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition", Interspeech, DOI: 10.21437/​Interspeech.2021-571, September 2021, pp. 726-730.
      BibTeX TR2021-103 PDF
      • @inproceedings{Higuchi2021sep,
      • author = {Higuchi, Yosuke and Moritz, Niko and Le Roux, Jonathan and Hori, Takaaki},
      • title = {Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition},
      • booktitle = {Interspeech},
      • year = 2021,
      • pages = {726--730},
      • month = sep,
      • doi = {10.21437/Interspeech.2021-571},
      • url = {https://www.merl.com/publications/TR2021-103}
      • }
  • MERL Contact:
  • Research Areas:

    Artificial Intelligence, Machine Learning, Speech & Audio

Abstract:

Pseudo-labeling (PL) has been shown to be effective in semi-supervised automatic speech recognition (ASR), where a base model is self-trained with pseudo-labels generated from unlabeled data. We present momentum pseudo-labeling (MPL), a simple yet effective strategy for semi-supervised ASR. MPL consists of a pair of online and offline models that interact and learn from each other, inspired by the mean teacher method.

 

  • Related Publication

  •  Higuchi, Y., Moritz, N., Le Roux, J., Hori, T., "Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition", arXiv, June 2021.
    BibTeX arXiv
    • @article{Higuchi2021jun,
    • author = {Higuchi, Yosuke and Moritz, Niko and Le Roux, Jonathan and Hori, Takaaki},
    • title = {Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition},
    • journal = {arXiv},
    • year = 2021,
    • month = jun,
    • url = {https://arxiv.org/abs/2106.08922}
    • }