LDA主题模型是近年来提出的一种具有文本表示能力的非监督学习模型。考虑到传统主题模型在处理大规模文本时存在的局限性，文中提出一种基于LDA模型的文本相似度计算方法。利用LDA为语料库建模，通过Gibbs抽样间接估算模型参数，将文本表示为固定隐含主题集上的概率分布，以此计算文本之间的相似度。最后将K-means算法作为文本相似度的评估指标。实验结果表明，与LSI模型相比，该方法能有效地提高文本相似度计算的准确性和文本聚类效果。%LDA topic model is an unsupervised model which exhibits superiority on latent topic modeling of text data in the research of re-cent years. Considering the disadvantage of the traditional topic model when dealing with the large-scale text corpuses,a method which improves text similarity computations by using LDA model is proposed. It models corpus with LDA,parameters are estimated with Gibbs sampling. Each document is represented for the probability distribution of fixed implied theme set and computed the similarity between the texts. Finally,the K-means algorithm is selected as the evaluation index of text similarity. Experimental results show this method can im-prove the accuracy of text similarity and clustering quality of text effectively compared with LSI model.
展开▼
