...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE transactions on multimedia >Unsupervised Video Summarization With Cycle-Consistent Adversarial LSTM Networks
【24h】

Unsupervised Video Summarization With Cycle-Consistent Adversarial LSTM Networks

机译：循环一致的对冲LSTM网络无监督的视频摘要

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Video summarization is an important technique to browse, manage and retrieve a large amount of videos efficiently. The main objective of video summarization is to minimize the information loss when selecting a subset of video frames from the original video, hence the summary video can faithfully represent the overall story of the original video. Recently developed unsupervised video summarization approaches are free of requiring tedious annotation on important frames to train a video summarization model and thus are practically attractive. However, their performance is still limited due to the difficulty of minimizing information loss between the summary and original videos. In this paper, we address unsupervised video summarization by developing a novel Cycle-consistent Adversarial LSTM architecture to effectively reduce the information loss in the summary video. The proposed model, named Cycle-SUM, consists of a frame selector and a cycle-consistent learning based evaluator. The selector is a bi-directional LSTM network to capture the long-range relationship between video frames. To overcome the difficulty of specifying a suitable information preserving metric between original video and summary video, the evaluator is introduced to "supervise" selector to improve the video summarization quality. Specifically, the evaluator is composed of two generative adversarial networks (GANs), in which the forward GAN component is learned to reconstruct the original video from summary video, while the backward GAN learns to invert the process. We establish the relation between mutual information maximization and such cycle learning procedure and further introduce cycle-consistent loss to regularize the summarization. Extensive experiments on three video summarization benchmark datasets demonstrate a state-of-the-art performance, and show the superiority of the Cycle-SUM model compared with other unsupervised approaches.
机译：视频摘要是有效浏览，管理和检索大量视频的重要技术。视频摘要的主要目的是在从原始视频选择视频帧的子集时最小化信息丢失，因此摘要视频可以忠实地代表原始视频的整体故事。最近开发了无监督的视频摘要方法，无需对培训视频概要模型的重要帧来说，繁琐的注释，因此实际上是有吸引力的。但是，由于难度最小化了摘要和原始视频之间的信息丢失，它们的性能仍然有限。在本文中，我们通过开发一种新的循环一致的对抗性LSTM架构来解决无监督的视频摘要，以有效地降低摘要视频中的信息丢失。所提出的模型，命名为周期 - 总和，包括帧选择器和基于周期一致的基于学习的评估器。选择器是一个双向LSTM网络，用于捕获视频帧之间的远程关系。为了克服在原始视频和摘要视频之间指定保存度量的难度，评估器被引入“监督”选择器以提高视频概括质量。具体地，评估者由两种生成的对抗性网络（GAN）组成，其中学习前向GaN组件来重建摘要视频的原始视频，而后退GaN学会反转该过程。我们建立了相互信息最大化与此类循环学习程序之间的关系，并进一步引入了周期一致的损失，以规范摘要。三个视频摘要基准数据集的广泛实验证明了最先进的性能，并与其他无监督的方法相比，循环和模型的优越性。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号