• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Neural Networks and Learning Systems, IEEE Transactions on >Convergence Analyses on On-Line Weight Noise Injection-Based Training Algorithms for MLPs
【24h】

Convergence Analyses on On-Line Weight Noise Injection-Based Training Algorithms for MLPs

机译：基于在线权重噪声注入的MLP训练算法的收敛性分析

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Injecting weight noise during training is a simple technique that has been proposed for almost two decades. However, little is known about its convergence behavior. This paper studies the convergence of two weight noise injection-based training algorithms, multiplicative weight noise injection with weight decay and additive weight noise injection with weight decay. We consider that they are applied to multilayer perceptrons either with linear or sigmoid output nodes. Let ${bf w}(t)$ be the weight vector, let $V({bf w})$ be the corresponding objective function of the training algorithm, let $alpha>0$ be the weight decay constant, and let $mu (t)$ be the step size. We show that if $mu (t)rightarrow 0$, then with probability one $E[Vert{bf w}(t)Vert_{2}^{2}]$ is bound and $lim_{trightarrowinfty}Vert{bf w}(t)Vert_{2}$ exists. Based on these two properties, we show that if $mu (t)rightarrow 0$, $sum_{t}mu (t)=infty$, and $sum_{t}mu (t)^{2}
机译：在训练过程中注入体重噪声是一项已经提出了将近二十年的简单技术。但是，对其收敛行为知之甚少。本文研究了两种基于权重噪声注入的训练算法的收敛性，即带权重衰减的乘性权重噪声注入和带权重衰减的加性权重噪声注入。我们认为它们适用于具有线性或S型输出节点的多层感知器。令$ {bf w}（t）$为权重向量，令$ V（{bf w}）$为训练算法的相应目标函数，令$ alpha> 0 $为权重衰减常数，令$ mu（t）$是步长。我们表明，如果$ mu（t）rightarrow 0 $，则有一个$ E [Vert {bf w}（t）Vert_ {2} ^ {2}] $绑定的可能性，而$ lim_ {trightarrowinfty} Vert {bf w }（t）Vert_ {2} $存在。根据这两个属性，我们表明，如果$ mu（t）rightarrow 0 $，$ sum_ {t} mu（t）= infty $和$ sum_ {t} mu（t）^ {2}

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号