• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Information Fusion >VADR: Discriminative Multimodal Explanations for Situational Understanding
【24h】

VADR: Discriminative Multimodal Explanations for Situational Understanding

机译：VADR：情境理解的多模态解释

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The focus of this paper is on the generation of multimodal explanations for information fusion tasks performed on multimodal data. We propose that separating modal components in saliency map explanations provides users with a better understanding of how convolutional neural networks process multimodal data. We adapt established state-of-the-art explainability techniques to mid-level fusion networks in order to better understand (a) which modality of the input contributes most to a model's decision and (b) which parts of the input data are most relevant to that decision. Our method separates temporal from non-temporal information to allow a user to focus their attention on salient elements of the scene that are changing in multiple modalities. The work is experimentally tested on an activity recognition task using video and audio data. In view of the fact that explanations need to be tailored to the type of user in a User Fusion context, we focus on meeting explanation requirements for system creators and operators respectively.
机译：本文的重点是针对多模式数据上执行的信息融合任务的多模式解释的生成。我们建议在显着性图说明中分离模态成分，为用户提供对卷积神经网络如何处理多模态数据的更好理解。我们将已建立的最新可解释性技术应用于中级融合网络，以便更好地理解（a）哪种输入方式对模型的决策贡献最大;（b）哪些输入数据的部分最相关这个决定。我们的方法将时间信息与非时间信息分开，以使用户将注意力集中在以多种方式变化的场景的显着元素上。使用视频和音频数据对一项活动识别任务进行了实验测试。鉴于在User Fusion上下文中需要针对用户类型量身定制解释，因此我们专注于分别满足系统创建者和操作员的解释要求。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号