首页> 外文期刊>Parallel and Distributed Systems, IEEE Transactions on >Design and Evaluation of Multiple-Level Data Staging for Blue Gene Systems
【24h】

Design and Evaluation of Multiple-Level Data Staging for Blue Gene Systems

机译:蓝色基因系统多级数据分级的设计和评估

获取原文
获取原文并翻译 | 示例
           

摘要

Parallel applications currently suffer from a significant imbalance between computational power and available I/O bandwidth. Additionally, the hierarchical organization of current Petascale systems contributes to an increase of the I/O subsystem latency. In these hierarchies, file access involves pipelining data through several networks with incremental latencies and higher probability of congestion. Future Exascale systems are likely to share this trait. This paper presents a scalable parallel I/O software system designed to transparently hide the latency of file system accesses to applications on these platforms. Our solution takes advantage of the hierarchy of networks involved in file accesses, to maximize the degree of overlap between computation, file I/O-related communication, and file system access. We describe and evaluate a two-level hierarchy for Blue Gene systems consisting of client-side and I/O node-side caching. Our file cache management modules coordinate the data staging between application and storage through the Blue Gene networks. The experimental results demonstrate that our architecture achieves significant performance improvements through a high degree of overlap between computation, communication, and file I/O.
机译:当前,并行应用程序在计算能力和可用I / O带宽之间存在严重的不平衡。另外,当前的Petascale系统的分层组织会导致I / O子系统延迟的增加。在这些层次结构中,文件访问涉及通过具有递增延迟和更高拥塞概率的多个网络对数据进行流水线处理。未来的百亿亿次计算机可能会共享此特征。本文提出了一种可扩展的并行I / O软件系统,旨在透明地隐藏文件系统访问这些平台上的应用程序的延迟。我们的解决方案利用了文件访问所涉及的网络层次结构,以最大程度地提高计算,文件I / O相关的通信和文件系统访问之间的重叠程度。我们描述和评估由客户端和I / O节点侧缓存组成的Blue Gene系统的两级层次结构。我们的文件缓存管理模块通过Blue Gene网络协调应用程序和存储之间的数据分段。实验结果表明,我们的体系结构通过在计算,通信和文件I / O之间高度重叠,实现了显着的性能改进。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号