• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Advanced Technologies for Signal and Image Processing >A comparison of big remote sensing data processing with Hadoop MapReduce and Spark
【24h】

A comparison of big remote sensing data processing with Hadoop MapReduce and Spark

机译：Hadoop MapReduce和Spark对大型遥感数据处理的比较

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The continuous generation of huge amount of remote sensing (RS) data is becoming a challenging task for researchers due to the 4 Vs characterizing this type of data (volume, variety, velocity and veracity). Many platforms have been proposed to deal with big data in RS field. This paper focus on the comparison of two well-known platforms of big RS data namely Hadoop and Spark. We start by describing the two platforms Hadoop and Spark. The first platform is designed for processing enormous unstructured data in a distributed computing environment. It is composed of two basic elements : 1) Hadoop Distributed file system for storage, and 2) Mapreduce and Yarn for parallel processing, scheduling the jobs and analyzing big RS data. The second platform, Spark, is composed by a set of libraries and uses the resilient distributed data set to overcome the computational complexity. The last part of this paper is devoted to a comparison between the two platforms.
机译：由于4 V代表这种类型的数据（体积，种类，速度和准确性），因此连续生成大量的遥感（RS）数据正成为研究人员的一项艰巨任务。已经提出了许多平台来处理RS领域中的大数据。本文着重比较两个著名的大RS数据平台Hadoop和Spark。我们首先描述Hadoop和Spark这两个平台。第一个平台设计用于在分布式计算环境中处理大量非结构化数据。它由两个基本元素组成：1）用于存储的Hadoop分布式文件系统，以及2）用于并行处理，调度作业和分析大RS数据的Mapreduce和Yarn。第二个平台Spark由一组库组成，并使用弹性分布式数据集来克服计算复杂性。本文的最后一部分致力于两个平台之间的比较。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号