• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Parallel and Distributed Systems >Towards Long-View Computing Load Balancing in Cluster Storage Systems
【24h】

Towards Long-View Computing Load Balancing in Cluster Storage Systems

机译：在集群存储系统中实现长远计算负载均衡

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

In large-scale computing clusters, when the server storing a task's requested data does not have sufficient computing capacity for the task, current job schedulers either schedule the task to the closest server and transmit to it the requested data, or let the task wait until the server has sufficient computing capacity. The former solution generates network load while the latter solution increases task delay. To handle this problem, load balancing methods are needed to reduce the number of overloaded servers due to computing workloads. However, current load balancing methods do not aim to balance the computing load for the long term. Through trace analysis, we demonstrate the diversity of computing workloads of different tasks and the necessity of balancing the computing workloads among servers. Then, we propose a cost-efficient Computing load Aware and Long-View load balancing approach (CALV ). CALV is novel in that it achieves long-term computing load balance by migrating out an overloaded server data blocks contributing more computing workloads when the server is more overloaded and contribute less computing workloads when the server is more underloaded at different epochs during a time period. Based upon the task schedules, we further propose a task reassignment algorithm that reassigns tasks from an overloaded server to other data servers of the tasks to make it non-overloaded before CALV is conducted. The above methods are for the tasks whose submission times and execution latencies can be predicted. To handle unexpected tasks or insufficiently accurate predictions, we propose a dynamic load balancing method, in which an overloaded server dynamically redirects tasks to other data servers of the tasks, or replicates the tasks’ requested data to other servers and redirects the tasks to those servers in order to become non-overloaded. Finally, we propose a proximity-aware tree based distributed load balancing method to reduce the reallocation cost and improve the scalability of CALV. Trace-driven experiments in simulation and a real computing cluster show that CALV outperforms other methods in terms of balancing the computing workloads and cost efficiency.
机译：在大型计算集群中，当存储任务的请求数据的服务器没有足够的计算能力来执行任务时，当前作业调度程序将任务调度到最近的服务器并向其传输请求的数据，或者让任务等待直到服务器具有足够的计算能力。前者会产生网络负载，而后者会增加任务延迟。为了解决此问题，需要使用负载平衡方法来减少由于计算工作负载而导致的服务器过载数量。但是，当前的负载平衡方法并不旨在长期平衡计算负载。通过跟踪分析，我们证明了不同任务的计算工作量的多样性以及在服务器之间平衡计算工作量的必要性。然后，我们提出了一种经济高效的计算负载感知和长视负载均衡方法（CALV）。 CALV的新颖之处在于，它可以通过迁移过载的服务器数据块来实现长期的计算负载平衡，当服务器负载更大时，该数据块将贡献更多的计算工作量，而当服务器在不同时期的负载量更低时，则将贡献更少的计算工作量。根据任务计划，我们进一步提出了一种任务重新分配算法，该任务可以在执行CALV之前将任务从过载的服务器重新分配给任务的其他数据服务器，以使其不过载。以上方法适用于可以预测提交时间和执行延迟的任务。为了处理意外任务或预测不够准确，我们提出了一种动态负载平衡方法，其中过载的服务器将任务动态地重定向到任务的其他数据服务器，或者将任务请求的数据复制到其他服务器，然后将任务重定向到那些服务器。为了变得不超载。最后，我们提出了一种基于邻近感知树的分布式负载均衡方法，以减少重新分配成本，提高CALV的可扩展性。在模拟和真实计算集群中进行跟踪驱动的实验表明，CALV在平衡计算工作量和成本效率方面优于其他方法。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号