• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Biological Information and Biomedical Engineering >Hand gesture based control with multi-modality data - towards surgical applications
【24h】

Hand gesture based control with multi-modality data - towards surgical applications

机译：用多种模式数据的手势基于多种模式数据 - 走向手术应用

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Image-guided surgery provides the surgeon with additional information to perform the operation more accurately. However, how to visualize the preoperative planning in an interactive way remains a challenge. Hand gesture based control provides touchless user interface thus a potential for surgical environment. Comparing to static hand gestures, dynamic gestures provide more natural way for human control. However, considering the complexity of dynamic hand gestures, there are challenges associated with sufficient feature extraction. In this paper, a multi-modality system was proposed for dynamic hand gesture recognition. Two types of features were defined including finger pose and hand motion, which were extracted by combining information from Leap Motion and depth images. A Long Short-Term Memory (LSTM) network was trained based on the extracted feature sequence for hand gesture recognition. Experiments were performed to evaluate the feasibility of the system with a 16-class dynamic gesture dataset, and an average accuracy of 94.10% was achieved. The results demonstrated that depth images can effectively compensate for feature missing due to lost tracking by Leap Motion, thus providing the possibility for surgical applications.
机译：图像引导的手术提供外科医生的附加信息，更准确地执行操作。但是，如何以互动方式可视化术前计划仍然是一个挑战。基于手势的控制提供了无情的用户界面，从而提供了外科环境的潜力。与静态手势相比，动态手势为人类控制提供了更自然的方式。然而，考虑到动态手势的复杂性，存在与足够特征提取相关的挑战。本文提出了一种用于动态手势识别的多模态系统。定义了两种类型的特征，包括通过组合来自跳跃运动和深度图像的信息来提取的手指姿势和手动运动。基于用于手势识别的提取的特征序列，训练了长短期存储器（LSTM）网络。进行实验以评估系统与16级动态手势数据集的可行性，实现了94.10％的平均精度。结果表明，深度图像可以有效地补偿由于跳跃运动丢失导致的特征缺失，从而提供了外科应用的可能性。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号