Previous studies proved that, adding part of speech tag information to the input layer of neural language model, can improve the performance significantly. But part of speech tag need hand-annotated data to train the tag model, which consumes a lot and the extra tagger also makes the model more complicated. To solve the problem, this article propose adding the results of brown clustering, instead of part of speech tag information to the input layer of the recurrent network language model. In the Penn Treebank corpus, the relative improvement over the original recurrent neural network language model reaches 8%~9%.%研究表明，在递归神经网络语言模型的输入层加入词性标注信息，可以显著提高模型的效果。但使用词性标注需要手工标注的数据训练，耗费大量的人力物力，并且额外的标注器增加了模型的复杂性。为了解决上述问题，本文尝试将布朗词聚类的结果代替词性标注信息加入到递归神经网络语言模型输入层。实验显示，在Penn Treebank语料上，加入布朗词类信息的递归神经网络语言模型相比原递归神经网络语言模型困惑度下降8~9%。
展开▼
