• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>Job scheduling strategies for parallel processing >Job Admission and Resource Allocation in Distributed Streaming Systems
【24h】

Job Admission and Resource Allocation in Distributed Streaming Systems

机译：分布式流系统中的工作准入和资源分配

获取原文
获取原文并翻译 | 示例
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

This paper describes a new and novel scheme for job admission and resource allocation employed by the SODA scheduler in System S. Capable of processing enormous quantities of streaming data, System S is a large-scale, distributed stream processing system designed to handle complex applications. The problem of scheduling in distributed, stream-based systems is quite unlike that in more traditional systems. And the requirements for System S, in particular, are more stringent than one might expect even in a "standard" stream-based design. For example, in System S, the offered load is expected to vastly exceed system capacity. So a careful job admission scheme is essential. The jobs in System S are essentially directed graphs, with software "processing elements" (PEs) as vertices and data streams as edges connecting the PEs. The jobs themselves are often heavily interconnected. Thus resource allocation of individual PEs must be done carefully in order to balance the flow. We describe the design of the SODA scheduler, with particular emphasis on the component, known as macroQ, which performs the job admission and resource allocation tasks. We demonstrate by experiments the natural trade-offs between job admission and resource allocation.
机译：本文介绍了SODA调度程序在System S中采用的一种新的作业接纳和资源分配方案。System S是能够处理大量流数据的大型大规模分布式流处理系统，旨在处理复杂的应用程序。分布式，基于流的系统中的调度问题与传统系统中的调度问题完全不同。尤其是对System S的要求，甚至在基于“标准”流的设计中，也比人们预期的要严格。例如，在系统S中，预期提供的负载将大大超过系统容量。因此，认真的工作录取计划至关重要。系统S中的作业本质上是有向图，软件“处理元素”（PE）作为顶点，数据流作为连接PE的边。这些工作本身通常是紧密相连的。因此，必须仔细进行各个PE的资源分配，以平衡流量。我们描述了SODA调度程序的设计，特别着重于称为macroQ的组件，该组件执行作业接纳和资源分配任务。我们通过实验证明了工作准入和资源分配之间的自然平衡。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号