• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Artificial Neural Networks >Residual Learning for FC Kernels of Convolutional Network
【24h】

Residual Learning for FC Kernels of Convolutional Network

机译：卷积网络FC核的残差学习

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

One of the most important steps in training a neural network is choosing its depth. Theoretically, it is possible to construct a complex decision-making function by cascading a number of shallow networks. It can produce a similar in accuracy result while providing a significant performance cost benefit. In practice, at some point, just increasing the depth of a network can actually decrease its performance due to over-learning. In literature, this is called "vanishing gradient descent". Vanishing gradient descent is observed as a vanishing decrease of magnitudes of gradients of weights for each subsequent layer, effectively preventing the weight from changing its value in the lower layers of a deep network when applying the backward propagation of errors algorithm. There is an approach called Residual Network (ResNet) to solve this problem for standard convolutional networks. However, the ResNet solves the problem only partially, as the resulting network is not sequential, but is an ensemble of shallow networks with all drawbacks typical for them. In this article, we investigate a convolutional network with fully connected layers (so-called network in network architecture, NiN) and suggest another way to build an ensemble of shallow networks. In our case, we gradually reduce the number of parallel connections by using sequential network connections. This allows to eliminate the influence of the vanishing gradient descent and to reduce the redundancy of the network by using all weight coefficients and not using residual blocks as ResNet does. For this method to work it is not required to change the network architecture, but only needed to properly initialize its weights.
机译：训练神经网络最重要的步骤之一就是选择其深度。从理论上讲，可以通过级联多个浅层网络来构建复杂的决策功能。它可以产生相似的精度结果，同时提供显着的性能成本优势。实际上，在某些时候，由于过度学习，仅增加网络的深度实际上会降低其性能。在文献中，这称为“消失梯度下降”。观察到消失梯度下降是每个后续层的权重梯度幅度的消失，从而有效地防止了权重在应用错误的向后传播算法时在深层网络的较低层中更改其值。有一种称为残差网络（ResNet）的方法可以解决标准卷积网络的此问题。但是，ResNet仅部分解决了该问题，因为生成的网络不是连续的，而是浅层网络的集合，具有所有它们典型的缺点。在本文中，我们研究了具有全连接层的卷积网络（在网络体系结构中称为网络NiN），并提出了另一种构建浅层网络的方法。在我们的情况下，我们通过使用顺序网络连接逐渐减少并行连接的数量。通过使用所有权重系数，而不像ResNet那样使用残差块，这可以消除消失的梯度下降的影响并减少网络的冗余。为了使该方法起作用，不需要更改网络体系结构，而仅需要适当地初始化其权重即可。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号