• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>IEEE International Conference on Dependable Systems, Services and Technologies >Non-Random Weight Initialisation in Deep Learning Networks for Repeatable Determinism
【24h】

Non-Random Weight Initialisation in Deep Learning Networks for Repeatable Determinism

机译：用于可重复决定论的深度学习网络中的非随机重量初始化

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

This research is examining the change in weight values of deep learning networks after learning. These research experiments require to make measurements and comparisons from a stable set of known weights and biases before and after learning is conducted, such that comparisons after learning are repeatable and the experiment is controlled. As such the current accepted schemes of random number initialisations of the weight values may need to be deterministic rather than stochastic to have little run to run varying effects, so that the weight value initialisations are not a varying contributor. This paper looks at the viability of non-random weight initialisation schemes, to be used in place of the random number weight initialisations of an established well understood test case. The viability of non-random weight initialisation schemes in neural networks may make a network more deterministic in learning sessions which is a desirable property in mission and safety critical systems. The paper will use a variety of schemes over number ranges and gradients and will achieve a 97.97% accuracy figure just 0.18% less than the original random number scheme at 98.05%. The paper may highlight that in this case it may be the number range and not the gradient that is effecting the achieved accuracy most dominantly, although there may be a coupling of number range with activation functions used. Unexpectedly in this paper, an effect of numerical instability will be discovered from run to run when run on a multi-core CPU. The paper will also show the enforcement of consistent deterministic results on an multi-core CPU by defining atomic critical code regions aiding repeatable Information Assurance (IA) in model fitting (or learning sessions).
机译：这项研究是学习考察后，深学习网络的权重值的变化。这些研究实验需要进行测量和比较，从之前稳定组已知重量和偏见和学习后进行，这样的学习后比较是可重复的，实验进行控制。这样的权重值的随机数initialisations目前接受的方案可能需要确定的，而不是随机的有一点在每次运行产生不同的影响，因此权重值为initialisations不是变贡献者。本文长相在非随机重量初始化方案的可行性，以代替已建立的很好理解的测试情况的​​随机数重量initialisations一起使用。非随机的重量初始化方案的神经网络生存能力可以使网络在学习会议是在使命和安全关键系统所期望的性能更具有确定性。本文将使用各种数量超过范围和梯度方案，将在98.05％，达到97.97％的准确数字比原来的随机数小于计划只是0.18％。纸张可突出显示，在这种情况下，它可以是数字范围和不是最显性实现所述获得的精确度梯度，虽然有可能是数范围与用于激活功能的耦合。意外地在本文中，数值不稳定性的效果将从运行中发现的多芯CPU上运行的运行时。此外，本文还将通过模型拟合确定原子关键代码区域协助重复信息保障（IA）（或学习会话）显示的多核CPU上一致的确定性结果的执行。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号