首页>
外国专利>
NEURAL NETWORK SYSTEM, NEURAL NETWORK TRAINING METHOD, AND NEURAL NETWORK TRAINING PROGRAM
NEURAL NETWORK SYSTEM, NEURAL NETWORK TRAINING METHOD, AND NEURAL NETWORK TRAINING PROGRAM
展开▼
机译：神经网络系统，神经网络训练方法和神经网络训练计划
展开▼
页面导航
摘要
著录项
相似文献
摘要
The present invention improves a throughput of data-parallel distributed training. This neural network system comprises a memory, and a plurality of processors that access the memory, wherein, in each of a plurality of trainings, each of the plurality of processors: executes a calculation of a neural network on the basis of an input of training data and parameters in the neural network, and calculates an output of the neural network; and calculates a gradient or an update amount based on the gradient for parameters of the difference between the calculated output and teacher data of the training data, wherein (1) in a first case where the accumulation of the gradient or the update amount is not smaller than a threshold, the plurality of processors execute a first update processing by respectively transmitting the accumulations of a plurality of the calculated gradients or update amounts to other processors among the plurality of processors to integrate the accumulations of the gradients or the update amounts, receiving the integrated accumulations of the gradients or the update amounts, and updating the parameters with the integrated accumulations of the gradients or the update amounts, and (2) in a second case where the accumulation of the gradient or the update amount is smaller than the threshold, the plurality of processors execute a second update processing by not integrating the plurality of accumulations of the gradients or the update amounts by the transmission, but respectively updating the parameters with the calculated gradients or update amounts.
展开▼
