...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Future generation computer systems >Flame-MR: An event-driven architecture for MapReduce applications
【24h】

Flame-MR: An event-driven architecture for MapReduce applications

机译：Flame-MR：适用于MapReduce应用程序的事件驱动架构

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Nowadays, many organizations analyze their data with the MapReduce paradigm, most of them using the popular Apache Hadoop framework. As the data size managed by MapReduce applications is steadily increasing, the need for improving the Hadoop performance also grows. Existing modifications of Hadoop (e.g., Mellanox Unstructured Data Accelerator) attempt to improve performance by changing some of its underlying subsystems. However, they are not always capable to cope with all its performance bottlenecks or they hinder its portability. Furthermore, new frameworks like Apache Spark or DataMPI can achieve good performance improvements, but they do not keep compatibility with existing MapReduce applications. This paper proposes Flame-MR, a new event-driven MapReduce architecture that increases Hadoop performance by avoiding memory copies and pipelining data movements, without modifying the source code of the applications. The performance evaluation on two representative systems (an HPC cluster and a public cloud platform) has shown experimental evidence of significant performance increases, reducing the execution time by up to 54% on the Amazon EC2 cloud.
机译：如今，许多组织使用MapReduce范式分析数据，其中大多数使用流行的Apache Hadoop框架。随着MapReduce应用程序管理的数据量稳定增长，对改善Hadoop性能的需求也随之增长。 Hadoop的现有修改版本（例如Mellanox非结构化数据加速器）试图通过更改其某些底层子系统来提高性能。但是，它们并不总是有能力应对其所有性能瓶颈，或者会阻碍其便携性。此外，诸如Apache Spark或DataMPI之类的新框架可以实现良好的性能改进，但它们无法与现有MapReduce应用程序保持兼容性。本文提出了Flame-MR，这是一种新的事件驱动的MapReduce架构，该架构通过避免内存复制和流水线化数据移动而无需修改应用程序的源代码，从而提高了Hadoop的性能。在两个代表性系统（HPC群集和公共云平台）上的性能评估显示出明显的性能提升的实验证据，从而使Amazon EC2云上的执行时间减少了多达54％。

著录项

  • 来源
    《Future generation computer systems》 |2016年第12期|46-56|共11页
  • 作者

    Jorge Veiga; Roberto R. Exposito; Guillermo L. Taboada; Juan Tourino;

  • 作者单位

    Grupo de Arquitectura de Computations (GAC), Departamento de Electronica e Sistemas, Facultade de Informatica, Universidade da Coruna, Campus de A Coruna, 15071 A Coruna, Spain;

    Grupo de Arquitectura de Computations (GAC), Departamento de Electronica e Sistemas, Facultade de Informatica, Universidade da Coruna, Campus de A Coruna, 15071 A Coruna, Spain;

    Grupo de Arquitectura de Computations (GAC), Departamento de Electronica e Sistemas, Facultade de Informatica, Universidade da Coruna, Campus de A Coruna, 15071 A Coruna, Spain;

    Grupo de Arquitectura de Computations (GAC), Departamento de Electronica e Sistemas, Facultade de Informatica, Universidade da Coruna, Campus de A Coruna, 15071 A Coruna, Spain;

  • 收录信息 美国《科学引文索引》(SCI);美国《工程索引》(EI);
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

    Big Data; MapReduce; Hadoop; Event-driven architecture; Cloud computing;

    机译：大数据;MapReduce;Hadoop;事件驱动架构;云计算;

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号