TR2015-033
Discriminative Method for Recurrent Neural Network Language Models
-
- "Discriminative Method for Recurrent Neural Network Language Models", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), DOI: 10.1109/ICASSP.2015.7179000, April 2015, pp. 5386-5390.BibTeX TR2015-033 PDF
- @inproceedings{Tachioka2015apr,
- author = {Tachioka, Y. and Watanabe, S.},
- title = {Discriminative Method for Recurrent Neural Network Language Models},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2015,
- pages = {5386--5390},
- month = apr,
- publisher = {IEEE},
- doi = {10.1109/ICASSP.2015.7179000},
- url = {https://www.merl.com/publications/TR2015-033}
- }
,
- "Discriminative Method for Recurrent Neural Network Language Models", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), DOI: 10.1109/ICASSP.2015.7179000, April 2015, pp. 5386-5390.
-
Research Areas:
Abstract:
A recurrent neural network language model (RNN-LM) can use a long word context more than can an n-gram language model, and its effective has recently been shown in its accomplishment of automatic speech recognition (ASR) tasks. However, the training criteria of RNN-LM are based on cross entropy (CE) between predicted and reference words. In addition, unlike the discriminative training of acoustic models and discriminative language models (DLM), these criteria do not explicitly consider discriminative criteria calculated from ASR hypotheses and references. This paper proposes a discriminative training method for RNN-LM by additionally considering a discriminative criterion to CE. We use the log-likelihood ratio of the ASR hypotheses and references as an discriminative criterion.
The proposed training criterion emphasizes the effect of misrecognized words relatively compared to the effect of correct words, which are discounted in training. Experiments on a large vocabulary continuous speech recognition task show that our proposed method improves the RNN-LM baseline. In addition, combining the proposed discriminative RNN-LM and DLM further shows its effectiveness.
Related News & Events
-
NEWS Nikkei reports on Mitsubishi Electric speech recognition Date: April 20, 2015Brief- Mitsubishi Electric researcher, Yuuki Tachioka of Japan, and MERL researcher, Shinji Watanabe, presented a paper at the IEEE International Conference on Acoustics, Speech & Signal Processing (ICASSP) entitled, "A Discriminative Method for Recurrent Neural Network Language Models". This paper describes a discriminative (language modelling) method for Japanese speech recognition. The Japanese Nikkei newspapers and some other press outlets reported on this method and its performance for Japanese speech recognition tasks.
-
NEWS Multimedia Group researchers presented 8 papers at ICASSP 2015 Date: April 19, 2015 - April 24, 2015
Where: IEEE International Conference on Acoustics, Speech & Signal Processing (ICASSP)
MERL Contacts: Anthony Vetro; Hassan Mansour; Petros T. Boufounos; Jonathan Le RouxBrief- Multimedia Group researchers have presented 8 papers at the recent IEEE International Conference on Acoustics, Speech & Signal Processing, which was held in Brisbane, Australia from April 19-24, 2015.