...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Journal of intelligent & fuzzy systems: Applications in Engineering and Technology >Highway II, an extended version of highway networks and its application to densely connected Bi-LSTM
【24h】

Highway II, an extended version of highway networks and its application to densely connected Bi-LSTM

机译：II高速公路II，扩展版的公路网络及其在密集连接的BI-LSTM中的应用

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The increase of depth is essential for the success of Deep Neural Networks while also leads to the difficulty of training. In light of this, the authors propose a novel multi-layer LSTM model called Highway-DC via introducing Highway Networks (Highway) to Densely Connected Bi-LSTM (DC-Bi-LSTM) which representation of each layer concatenates the output of itself and all preceding layers. Highway is applied to control the volume of input or output of each layer in DC-Bi-LSTM to the next. However, results reveal that Highway-DC shows no improvement over DC-Bi-LSTM, thus an extended version of Highway named Highway II is proposed via eliminating the multiplicative connections between transform gate and the output in Highway thus preserve the learning of each layer. And the Highway II-based model is named Highway II-DC. Evaluated on 7 benchmark datasets of text classification with compare to DC-Bi-LSTM and other state-of-the-art approaches, results indicate that Highway II-DC shows promising performance for achieving state-of-the-art on 3 datasets and surpassing DC-Bi-LSTM on 6 datasets with faster speed to converge. Besides, it can still enjoy the gain of increased layers with depth up to 30, while DC-Bi-LSTM gets saturated early at a depth of 15.
机译：深度的增加对于深度神经网络的成功是必不可少的，同时也导致训练难度。鉴于此，作者提出了一种新的多层LSTM模型，称为高速公路DC，通过将公路网络（公路）引入密集连接的Bi-LSTM（DC-Bi-LSTM），每个层表示其自身的输出和所有前面的层。应用高速公路以控制DC-BI-LSTM中每层的输入或输出的量。然而，结果表明，高速公路-DC显示了DC-BI-LSTM的改进，因此通过消除转换门和高速公路输出之间的乘法连接，提出了名为Highway II的扩展版本的高速公路，从而保留了每层的学习。基于II的高速公路模型名为II-DC。在与DC-BI-LSTM和其他最先进的方法比较的文本分类上评估了7个基准数据集，结果表明，II-DC高速公路显示出在3个数据集上实现最先进的表现超过6个数据集的DC-BI-LSTM，速度更快地收敛。此外，它仍然可以享受增加的层，深度增加30，而DC-BI-LSTM在深度的深度升高。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号