• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Joint Conference on Neural Networks >Hardware-driven nonlinear activation for stochastic computing based deep convolutional neural networks
【24h】

Hardware-driven nonlinear activation for stochastic computing based deep convolutional neural networks

机译：基于硬件驱动的非线性激活的基于深度卷积神经网络的随机计算

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Recently, Deep Convolutional Neural Networks (DCNNs) have made unprecedented progress, achieving the accuracy close to, or even better than human-level perception in various tasks. There is a timely need to map the latest software DCNNs to application-specific hardware, in order to achieve orders of magnitude improvement in performance, energy efficiency and compactness. Stochastic Computing (SC), as a low-cost alternative to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementation of DCNNs. One major challenge in SC based DCNNs is designing accurate nonlinear activation functions, which have a significant impact on the network-level accuracy but cannot be implemented accurately by existing SC computing blocks. In this paper, we design and optimize SC based neurons, and we propose highly accurate activation designs for the three most frequently used activation functions in software DCNNs, i.e, hyperbolic tangent, logistic, and rectified linear units. Experimental results on LeNet-5 using MNIST dataset demonstrate that compared with a binary ASIC hardware DCNN, the DCNN with the proposed SC neurons can achieve up to 61X, 151X, and 2X improvement in terms of area, power, and energy, respectively, at the cost of small precision degradation. In addition, the SC approach achieves up to 21X and 41X of the area, 41X and 72X of the power, and 198200X and 96443X of the energy, compared with CPU and GPU approaches, respectively, while the error is increased by less than 3.07%. ReLU activation is suggested for future SC based DCNNs considering its superior performance under a small bit stream length.
机译：最近，深度卷积神经网络（DCNN）取得了空前的进步，在各种任务中达到了接近或什至优于人类感知的精度。迫切需要将最新的软件DCNN映射到特定于应用程序的硬件，以实现性能，能效和紧凑性的数量级改进。随机计算（SC）作为传统二进制计算范例的一种低成本替代品，具有实现DCNN大规模并行且高度可扩展的硬件实现的潜力。基于SC的DCNN的一个主要挑战是设计精确的非线性激活函数，该函数对网络级的精度有重大影响，但是无法通过现有的SC计算模块准确实现。在本文中，我们设计和优化了基于SC的神经元，并且针对软件DCNN中三个最常用的激活函数（即双曲正切，逻辑和精简线性单元）提出了高度精确的激活设计。使用MNIST数据集在LeNet-5上进行的实验结果表明，与二进制ASIC硬件DCNN相比，带有拟议中的SC神经元的DCNN在面积，功率和能量方面分别可分别提高61倍，151倍和2倍。精度下降的代价很小。此外，与CPU和GPU方法相比，SC方法分别实现了高达21X和41X的面积，41X和72X的能量以及198200X和96443X的能量，而误差却增加了不到3.07 \ ％。考虑到其在小比特流长度下的优越性能，建议将其用于未来基于SC的DCNN的ReLU激活。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号