• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>International Journal of High Performance Computing Applications >Modeling high-throughput applications for in situ analytics
【24h】

Modeling high-throughput applications for in situ analytics

机译：为高通量应用程序建模以进行原位分析

获取原文
获取原文并翻译 | 示例
       
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

With the goal of performing exascale computing, the importance of input/output (I/O) management becomes more and more critical to maintain system performance. While the computing capacities of machines are getting higher, the I/O capabilities of systems do not increase as fast. We are able to generate more data but unable to manage them efficiently due to variability of I/O performance. Limiting the requests to the parallel file system (PFS) becomes necessary. To address this issue, new strategies are being developed such as online in situ analysis. The idea is to overcome the limitations of basic postmortem data analysis where the data have to be stored on PFS first and processed later. There are several software solutions that allow users to specifically dedicate nodes for analysis of data and distribute the computation tasks over different sets of nodes. Thus far, they rely on a manual resource partitioning and allocation by the user of tasks (simulations, analysis). In this work, we propose a memory-constraint modelization for in situ analysis. We use this model to provide different scheduling policies to determine both the number of resources that should be dedicated to analysis functions and that schedule efficiently these functions. We evaluate them and show the importance of considering memory constraints in the model. Finally, we discuss the different challenges that have to be addressed to build automatic tools for in situ analytics.
机译：以执行百亿亿次计算为目标，输入/输出（I / O）管理的重要性对于维持系统性能变得越来越重要。在机器的计算能力越来越高的同时，系统的I / O能力却没有迅速增加。由于I / O性能的变化，我们能够生成更多数据，但无法有效地管理它们。必须将请求限制在并行文件系统（PFS）中。为了解决这个问题，正在开发新的策略，例如在线原位分析。这个想法是要克服基本验尸数据分析的局限性，在这种情况下，数据必须先存储在PFS上，然后再进行处理。有几种软件解决方案可让用户专门指定用于数据分析的节点，并将计算任务分布在不同的节点集上。到目前为止，它们依赖于用户对任务（模拟，分析）的手动资源分配和分配。在这项工作中，我们提出了一种用于原位分析的内存约束模型。我们使用此模型提供不同的调度策略，以确定既应专用于分析功能的资源数量，又应确定有效调度这些功能的资源。我们对其进行了评估，并显示了在模型中考虑内存约束的重要性。最后，我们讨论了构建用于现场分析的自动工具所必须解决的不同挑战。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号