• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>Conference on the North American Chapter of the Association for Computational Linguistics: Human Language Technologies >Ricmannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
【24h】

Ricmannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling

机译：ricmannian在变分的Wasserstein AutoEncoder上的正常化文本建模

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Recurrent Variational Autoencoder has been widely used for language modeling and text generation tasks. These models often face a difficult optimization problem, also known as the Kullback-Leibler (KL) term vanishing issue, where the posterior easily collapses to the prior, and the model will ignore latent codes in generative tasks. To address this problem, we introduce an improved Wasserstein Variational Autoencoder (WAE) with Ricmannian Normalizing Flow (RNF) for text modeling. The RNF transforms a latent variable into a space that respecls the geometric characteristics of input space, which makes posterior impossible to collapse to the non-informative prior. The Wasserstein objective minimizes the distance between the marginal distribution and the prior directly, and therefore does not force the posterior to match the prior. Empirical experiments show that our model avoids KL vanishing over a range of datasets and has better performances in tasks such as language modeling, likelihood approximation, and text generation. Through a series of experiments and analysis over latent space, we show that our model learns latent distributions that respect latent space geometry and is able to generate sentences that are more diverse.
机译：经常性变化性AutoEncoder已广泛用于语言建模和文本生成任务。这些模型经常面临困难的优化问题，也称为Kullback-Leibler（KL）术语消失问题，其中后续容易折叠到先前，并且该模型将忽略生成任务中的潜在代码。为了解决这个问题，我们介绍了一种改进的Wassersein变分AutoEncoder（WAE），具有Ricmannian标准化流量（RNF），用于文本建模。 RNF将潜在变量转换为respecls输入空间的几何特征的空间，这使得后部不可能折叠到非信息性之前。 Wasserstein目的最小化边缘分布与之前的距离，因此不会强制后部匹配先前。实证实验表明，我们的模型避免了在一系列数据集中消失的KL，并且在语言建模，似然逼近和文本生成之类的任务中具有更好的性能。通过一系列对潜在空间的实验和分析，我们表明我们的模型了解尊重潜在空间几何形状的潜在分布，并且能够生成更多样化的句子。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号