• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Neural Networks, IEEE Transactions on >Objective Functions of Online Weight Noise Injection Training Algorithms for MLPs
【24h】

Objective Functions of Online Weight Noise Injection Training Algorithms for MLPs

机译：MLP在线权重噪声注入训练算法的目标函数

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Injecting weight noise during training has been a simple strategy to improve the fault tolerance of multilayer perceptrons (MLPs) for almost two decades, and several online training algorithms have been proposed in this regard. However, there are some misconceptions about the objective functions being minimized by these algorithms. Some existing results misinterpret that the prediction error of a trained MLP affected by weight noise is equivalent to the objective function of a weight noise injection algorithm. In this brief, we would like to clarify these misconceptions. Two weight noise injection scenarios will be considered: one is based on additive weight noise injection and the other is based on multiplicative weight noise injection. To avoid the misconceptions, we use their mean updating equations to analyze the objective functions. For injecting additive weight noise during training, we show that the true objective function is identical to the prediction error of a faulty MLP whose weights are affected by additive weight noise. It consists of the conventional mean square error and a smoothing regularizer. For injecting multiplicative weight noise during training, we show that the objective function is different from the prediction error of a faulty MLP whose weights are affected by multiplicative weight noise. With our results, some existing misconceptions regarding MLP training with weight noise injection can now be resolved.
机译：在训练过程中注入体重噪声一直是提高多层感知器（MLP）容错能力的简单策略，并且在这方面已提出了几种在线训练算法。但是，对于这些算法将目标函数最小化存在一些误解。一些现有的结果误解为受权重噪声影响的训练MLP的预测误差等同于权重噪声注入算法的目标函数。在本摘要中，我们想澄清这些误解。将考虑两种权重噪声注入方案：一种是基于加法权重噪声注入，另一种是基于乘性权重噪声注入。为了避免误解，我们使用它们的均值更新方程来分析目标函数。为了在训练过程中注入附加重量噪声，我们证明了真实的目标函数与故障MLP的预测误差相同，该故障MLP的重量受附加重量噪声的影响。它由常规的均方误差和平滑正则化器组成。对于在训练过程中注入乘性权重噪声，我们证明了目标函数不同于故障MLP的预测误差，该故障MLP的权重受乘性权重噪声影响。根据我们的结果，现在可以解决一些有关使用体重噪声注射进行MLP训练的误解。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号