• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International conference on machine vision >From Image Captioning to Video Summary using Deep Recurrent Networks and Unsupervised Segmentation
【24h】

From Image Captioning to Video Summary using Deep Recurrent Networks and Unsupervised Segmentation

机译：使用深度递归网络和无监督分割从图像字幕到视频摘要

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Automatic captioning systems based on recurrent neural networks have been tremendously successful at providing realistic natural language captions for complex and varied image data. We explore methods for adapting existing models trained on large image caption data sets to a similar problem, that of summarising videos using natural language descriptions and frame selection. These architectures create internal high level representations of the input image that can be used to define probability distributions and distance metrics on these distributions. Specifically, we interpret each hidden unit inside a layer of the caption model as representing the un-normalised log probability of some unknown image feature of interest for the caption generation process. We can then apply well understood statistical divergence measures to express the difference between images and create an unsupervised segmentation of video frames, classifying consecutive images of low divergence as belonging to the same context, and those of high divergence as belonging to different contexts. To provide a final summary of the video, we provide a group of selected frames and a text description accompanying them, allowing a user to perform a quick exploration of large unlabeled video databases.
机译：基于递归神经网络的自动字幕系统在为复杂多样的图像数据提供逼真的自然语言字幕方面已经取得了巨大的成功。我们探索使在大型图像字幕数据集上训练的现有模型适应类似问题的方法，即使用自然语言描述和帧选择对视频进行汇总的方法。这些体系结构创建了输入图像的内部高级表示形式，可用于定义概率分布和这些分布上的距离度量。具体来说，我们将字幕模型层内的每个隐藏单元解释为表示字幕生成过程中感兴趣的某些未知图像特征的未归一化对数概率。然后，我们可以应用易于理解的统计差异度量来表达图像之间的差异，并创建视频帧的无监督分割，将低散度的连续图像归为同一上下文，将高散度的连续图像归为不同上下文。为了提供视频的最终摘要，我们提供了一组选定的帧以及伴随它们的文本描述，从而使用户可以快速浏览大型的未标记视频数据库。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号