• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Parallel and Distributed Systems >RAMSYS: Resource-Aware Asynchronous Data Transfer with Multicore SYStems
【24h】

RAMSYS: Resource-Aware Asynchronous Data Transfer with Multicore SYStems

机译：RAMSYS：具有多核系统的资源感知异步数据传输

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

High-speed data transfer is vital to data-intensive computing that often requires moving large data volumes efficiently within a local data center and among geographically dispersed facilities. Effective utilization of the abundant resources in modern multicore environments for data transfer remains a persistent challenge, particularly, for Non-Uniform Memory Access (NUMA) systems wherein the locality of data accessing is an important factor. This requires rethinking how to exploit parallel access to data and to optimize the storage and network I/Os. We address this challenge and present a novel design of asynchronous processing and resource-aware task scheduling in the context of high-throughput data replication. Our software allocates multiple sets of threads to different stages of the processing pipeline, including storage I/O and network communication, based on their capacities. Threads belonging to each stage follow an asynchronous model, and attain high performance via multiple locality-aware and peer-aware mechanisms, such as task grouping, buffer sharing, affinity control and communication protocols. Our design also integrates high performance features to enhance the scalability of data transfer in several scenarios, e.g., file-level sorting, block-level asynchrony, and thread-level pipelining. Our experiments confirm the advantages of our software under different types of workloads and dynamic environments with contention for shared resources, including a 28-160 percent increase in bandwidth for transferring large files, 1.7-66 times speed-up for small files, and up to 108 percent larger throughput for mixed workloads compared with three state of the art alternatives, GridFTP , BBCP and Aspera.
机译：高速数据传输对于数据密集型计算至关重要，而数据密集型计算通常需要在本地数据中心内以及地理位置分散的设施之间有效地移动大量数据。在现代多核环境中如何有效利用丰富的资源进行数据传输仍然是一项持续的挑战，特别是对于非统一内存访问（NUMA）系统，其中数据访问的本地性是一个重要因素。这需要重新考虑如何利用对数据的并行访问以及如何优化存储和网络I / O。我们解决了这一挑战，并提出了一种在高通量数据复制的情况下进行异步处理和资源感知任务调度的新颖设计。我们的软件根据容量将多组线程分配给处理管道的不同阶段，包括存储I / O和网络通信。属于每个阶段的线程遵循异步模型，并通过多种本地感知和对等感知机制（例如任务分组，缓冲区共享，亲和力控制和通信协议）来获得高性能。我们的设计还集成了高性能功能，以在几种情况下增强数据传输的可伸缩性，例如文件级排序，块级异步和线程级流水线。我们的实验证实了我们的软件在不同类型的工作负载和动态环境下的优势，可以争用共享资源，包括传输大型文件的带宽增加28-160％，小型文件的加速1.7-66倍以及与三种最先进的替代方案GridFTP，BBCP和Aspera相比，混合工作负载的吞吐量提高了108％。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号