• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Experimental Mechanics >Big data and extreme-scale computing: Pathways to Convergence-Toward a shaping strategy for a future software and data ecosystem for scientific inquiry
【24h】

Big data and extreme-scale computing: Pathways to Convergence-Toward a shaping strategy for a future software and data ecosystem for scientific inquiry

机译：大数据和超大规模计算：融合之路-制定未来科学探索软件和数据生态系统的塑造策略

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Over the past four years, the Big Data and Exascale Computing (BDEC) project organized a series of five international workshops that aimed to explore the ways in which the new forms of data-centric discovery introduced by the ongoing revolution in high-end data analysis (HDA) might be integrated with the established, simulation-centric paradigm of the high-performance computing (HPC) community. Based on those meetings, we argue that the rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analyzing and using that data are radically reshaping the landscape of scientific computing. The most critical problems involve the logistics of wide-area, multistage workflows that will move back and forth across the computing continuum, between the multitude of distributed sensors, instruments and other devices at the networks edge, and the centralized resources of commercial clouds and HPC centers. We suggest that the prospects for the future integration of technological infrastructures and research ecosystems need to be considered at three different levels. First, we discuss the convergence of research applications and workflows that establish a research paradigm that combines both HPC and HDA, where ongoing progress is already motivating efforts at the other two levels. Second, we offer an account of some of the problems involved with creating a converged infrastructure for peripheral environments, that is, a shared infrastructure that can be deployed throughout the network in a scalable manner to meet the highly diverse requirements for processing, communication, and buffering/storage of massive data workflows of many different scientific domains. Third, we focus on some opportunities for software ecosystem convergence in big, logically centralized facilities that execute large-scale simulations and models and/or perform large-scale data analytics. We close by offering some conclusions and recommendations for future investment and policy review.
机译：在过去的四年中，大数据和百亿亿次计算（BDEC）项目组织了一系列的五个国际研讨会，旨在探讨高端数据分析的持续革命所引入的以数据为中心的新形式的发现方式。 （HDA）可能与高性能计算（HPC）社区已建立的，以仿真为中心的范例集成在一起。在这些会议的基础上，我们认为数字数据生成器的迅速普及，其生成数据的数量和多样性的空前增长以及分析和使用该数据的方法的迅猛发展正在从根本上改变科学的格局。计算。最关键的问题涉及广域，多阶段工作流的物流，这些工作流将在计算连续性，网络边缘的众多分布式传感器，仪器和其他设备之间以及商业云和HPC的集中式资源之间来回移动中心。我们建议，需要在三个不同的层面上考虑技术基础设施和研究生态系统未来集成的前景。首先，我们讨论研究应用程序和工作流的融合，以建立结合HPC和HDA的研究范例，其中不断取得的进步已经在其他两个层次上激发了人们的努力。其次，我们介绍了为外围环境创建融合基础架构所涉及的一些问题，即可以以可伸缩方式在整个网络中部署的共享基础架构，以满足处理，通信和通信的高度多样化的要求。缓冲/存储许多不同科学领域的海量数据工作流。第三，我们关注在大型逻辑集中的设施中软件生态系统融合的一些机会，这些设施执行大规模仿真和模型和/或执行大规模数据分析。最后，我们提供一些结论和建议，以供将来进行投资和政策审查之用。

著录项

  • 来源
    《Experimental Mechanics》 |2018年第4期|435-479|共45页
  • 作者

    Asch M.; Moore T.; Badia R.; Beck M.; Beckman P.; Bidot T.; Bodin F.; Cappello F.; Choudhary A.; de Supinski B.; Deelman E.; Dongarra J.; Dubey A.; Fox G.; Fu H.; Girona S.; Gropp W.; Heroux M.; Ishikawa Y.; Keahey K.; Keyes D.; Kramer W.; Lavignon J-F; Lu Y.; Matsuoka S.; Mohr B.; Reed D.; Requena S.; Saltz J.; Schulthess T.; Stevens R.; Swany M.; Szalay A.; Tang W.; Varoquaux G.; Vilotte J-P; Wisniewski R.; Xu Z.; Zacharov I.;

  • 作者单位

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

    Univ Tennessee, Innovat Comp Lab, Knoxville, TN USA;

  • 收录信息 美国《科学引文索引》(SCI);美国《工程索引》(EI);
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

    Big data; extreme-scale computing; future software; traditional HPC; high-end data analysis;

    机译：大数据;超大规模计算;未来软件;传统HPC;高端数据分析;

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号