• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Parallel and Distributed Systems >Preemptive and Low Latency Datacenter Scheduling via Lightweight Containers
【24h】

Preemptive and Low Latency Datacenter Scheduling via Lightweight Containers

机译：通过轻量级容器调度抢先和低延迟数据中心

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Datacenters are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve higher resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and QoS constraints. On one hand, latency-critical jobs need to be scheduled as soon as they are submitted to avoid any queuing delays. On the other hand, best-effort long jobs should be allowed to occupy the cluster when there are idle resources to improve cluster utilization. The challenge lies in how to minimize the queuing delays of short jobs while maximizing cluster utilization. In this article, we propose and develop BIG-C, a container-based resource management framework for data-intensive cluster computing. The key design is to leverage lightweight virtualization, a.k.a, containers, to make tasks preemptable in cluster scheduling. We devise two types of preemption strategies: immediate and graceful preemptions and show their effectiveness and tradeoffs with loosely-coupled MapReduce workloads as well as iterative, in-memory Spark workloads. Based on the mechanisms for task preemption, we further develop job-level and task-level preemptive policies as well as a preemptive fair share cluster scheduler. Our implementation on Yarn and evaluation with synthetic and production workloads show that low job latency and high resource utilization can be both attained when scheduling heterogeneous workloads on a contended cluster.
机译：数据中心正在发展到共享集群上的异构工作负载，以降低运营成本并实现更高的资源利用率。但是，将异构工作负载与各种资源需求和QoS约束进行了挑战。一方面，一方面需要在提交后立即安排延迟关键工作以避免任何排队延迟。另一方面，当存在空闲资源以提高集群利用时，应允许最佳努力占用群集。挑战在于如何最大限度地降低短岗位的排队延迟，同时最大限度地利用群集利用率。在本文中，我们提出并开发了BIG-C，是一种基于集装箱的资源管理框架，用于数据密集型集群计算。关键设计是利用轻量级虚拟化，A.K.A，容器，使任务在集群调度中抢占。我们设计了两种类型的抢占策略：立即和优雅的抢先，并展示了它们的效力和权衡，具有松散耦合的MapReduce工作负载以及迭代，内存的火花工作负载。根据任务抢占机制，我们进一步开发工作级别和任务级抢先政策以及先发制人的公平股票群集调度程序。我们对纱线的实施以及合成和生产工作负载的评估表明，在调度所搏斗的集群上的异构工作负载时，可以获得低作业延迟和高资源利用率。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号