• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>International Journal of Multimedia Information Retrieval >Characterization and classification of semantic image-text relations
【24h】

Characterization and classification of semantic image-text relations

机译：语义图像文本关系的特征与分类

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The beneficial, complementary nature of visual and textual information to convey information is widely known, for example, in entertainment, news, advertisements, science, or education. While the complex interplay of image and text to form semantic meaning has been thoroughly studied in linguistics and communication sciences for several decades, computer vision and multimedia research remained on the surface of the problem more or less. An exception is previous work that introduced the two metrics Cross-Modal Mutual Information and Semantic Correlation in order to model complex image-text relations. In this paper, we motivate the necessity of an additional metric called Status in order to cover complex image-text relations more completely. This set of metrics enables us to derive a novel categorization of eight semantic image-text classes based on three dimensions. In addition, we demonstrate how to automatically gather and augment a dataset for these classes from the Web. Further, we present a deep learning system to automatically predict either of the three metrics, as well as a system to directly predict the eight image-text classes. Experimental results show the feasibility of the approach, whereby the predict-all approach outperforms the cascaded approach of the metric classifiers.
机译：例如，在娱乐，新闻，广告，科学或教育中众所周知，传达信息的有益的，互补性的性质广为人知。虽然图像和文本的复杂相互作用在语言学和通信科学中彻底研究了几十年来，但计算机愿景和多媒体研究仍然在问题的表面上或多或少。例外是先前的工作，引入了两个度量跨模型互信息和语义相关性，以便模拟复杂的图像文本关系。在本文中，我们激励了额外的公制称为状态的必要性，以便更完整地涵盖复杂的图像文本关系。这组指标使我们能够基于三个维度导出八个语义图像文本类的新颖分类。此外，我们展示了如何从Web上自动收集和增强数据集。此外，我们展示了一个深度学习系统，可以自动预测三个度量标准，以及直接预测八个图像文本类的系统。实验结果表明该方法的可行性，从而预测 - 所有方法都优于公制分类器的级联方法。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号