动作预测是一类特殊的动作识别问题，不同于针对完整动作的传统动作识别，动作预测旨在动作尚未完成时尽可能早地识别动作所属的类别，以便对该动作可能造成的影响进行分析，从而实现事故预警、智能陪护、犯罪预警等目标。本文针对实时动作预测问题提出一种应用知识蒸馏技术的多阶段LSTM实时动作预测方法。本文中的动作预测模型为两阶段的LSTM模型，在第一阶段利用全局特征对动作进行分析，第二阶段利用全局特征与动作特征对动作进行分析。为提高动作预测模型的性能，本文利用知识蒸馏技术并设计新型的损失函数提高动作预测模型的性能。UT-Interaction数据集、JHMDB-21数据集以及UCF-101数据集的实验结果表明本文所提出的动作预测方法不但具有良好的动作预测能力，而且能够满足实际应用中的实时性要求。 Action recognition is a hot topic in the domain of computer vision, and it’s widely applied in human-computer interaction, studio entertainment, automatic drive, intelligent video surveillance, and intelligent medical care. Action prediction is a special class of action recognition. Different from conventional action recognition which aims at recognizing complete actions, the purpose of action prediction is to distinguish an action before it’s fully executed so that some objectives, such as accident early warning and crime prevention, can be achieved by analyzing the possible impact of the action. In order to solve the problem of real-time action prediction, this paper develops a multi-stage LSTM architecture that leverages knowledge distillation technique. The context-aware fea-ture and action-aware feature are exploited for action modeling. The proposed multi-stage LSTM architecture is composed of two stages. In the first stage it focuses on the global, context-aware information. The second stage then combines these context-aware features with action-aware ones. In order to improve the performance of proposed method in the early stage, the knowledge distillation technique is exploited for transferring the knowledge from teacher model to student model. A novel loss function is designed for the whole action prediction architecture and the performance is improved with the novel loss function. Experimental results on the UT-Interaction dataset, JHMDB-21 dataset and the UCF-101 dataset show that the proposed methods not only improve the accuracy of action prediction but also have the ability of real-time running.
展开▼
机译：动作预测是一类特殊的动作识别问题，不同于针对完整动作的传统动作识别，动作预测旨在动作尚未完成时尽可能早地识别动作所属的类别，以便对该动作可能造成的影响进行分析，从而实现事故预警、智能陪护、犯罪预警等目标。本文针对实时动作预测问题提出一种应用知识蒸馏技术的多阶段LSTM实时动作预测方法。本文中的动作预测模型为两阶段的LSTM模型，在第一阶段利用全局特征对动作进行分析，第二阶段利用全局特征与动作特征对动作进行分析。为提高动作预测模型的性能，本文利用知识蒸馏技术并设计新型的损失函数提高动作预测模型的性能。UT-Interaction数据集、JHMDB-21数据集以及UCF-101数据集的实验结果表明本文所提出的动作预测方法不但具有良好的动作预测能力，而且能够满足实际应用中的实时性要求。 Action recognition is a hot topic in the domain of computer vision, and it’s widely applied in human-computer interaction, studio entertainment, automatic drive, intelligent video surveillance, and intelligent medical care. Action prediction is a special class of action recognition. Different from conventional action recognition which aims at recognizing complete actions, the purpose of action prediction is to distinguish an action before it’s fully executed so that some objectives, such as accident early warning and crime prevention, can be achieved by analyzing the possible impact of the action. In order to solve the problem of real-time action prediction, this paper develops a multi-stage LSTM architecture that leverages knowledge distillation technique. The context-aware fea-ture and action-aware feature are exploited for action modeling. The proposed multi-stage LSTM architecture is composed of two stages. In the first stage it focuses on the global, context-aware information. The second stage then combines these context-aware features with action-aware ones. In order to improve the performance of proposed method in the early stage, the knowledge distillation technique is exploited for transferring the knowledge from teacher model to student model. A novel loss function is designed for the whole action prediction architecture and the performance is improved with the novel loss function. Experimental results on the UT-Interaction dataset, JHMDB-21 dataset and the UCF-101 dataset show that the proposed methods not only improve the accuracy of action prediction but also have the ability of real-time running.
展开▼
