• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Services Computing, IEEE Transactions on >Transformation-Based Streaming Workflow Allocation on Geo-Distributed Datacenters for Streaming Big Data Processing
【24h】

Transformation-Based Streaming Workflow Allocation on Geo-Distributed Datacenters for Streaming Big Data Processing

机译：地理分布数据中心上基于转换的流工作流分配，用于流式处理大数据

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The cost-minimization problem for streaming workflow (SW) has already become increasingly important and even critical in stream big data processing, particularly for geographically distributed datacenters, because of its huge demand on computing and communicating resources. Existing virtual machine (VM) allocation algorithms in cloud computing have been widely applied to batch-processing models; however, none of them can be successfully applied to SW because: 1) they failed to adapt the continuous execution characteristic of SW; and 2) most of them are all based on the assumption that the price of traffic and VMs among datacenters are uniform. In this paper, we propose a transformation-based SW allocation algorithm with the goal of cost-minimization for stream big data processing in geographically distributed datacenters, considering the characteristics of SW and price heterogeneity among geographically distributed datacenters. We first propose a cost-aware workflow transformation framework based on eight well-designed and verified transformation rules for cost reduction to adapt the continuous execution characteristic of SW. We then formulate the joint VM-traffic optimization problem and show that it is NP-hard. To produce the optimal solution in polynomial time, we then transform the SW allocation problem into the minimum-cost maximum-flow problem, considering both traffic and VMs price heterogeneity. Finally, our experimental results validate the high cost efficiency of our approach with lower computing and communicating costs by optimizing the workflow specification and joint VM-traffic cost optimization.
机译：由于对计算和通信资源的巨大需求，流工作流（SW）的成本最小化问题在流大数据处理（尤其是对于地理分布的数据中心）中已变得越来越重要，甚至变得至关重要。云计算中现有的虚拟机（VM）分配算法已广泛应用于批处理模型；但是，它们都不能成功地应用于SW，因为：1）它们不能适应SW的连续执行特性； 2）大多数都是基于这样的假设，即数据中心之间的流量和VM的价格是统一的。在本文中，我们考虑到地理分布数据中心之间的SW的特点和价格异质性，提出了一种基于转换的SW分配算法，其目的是最小化地理分布数据中心中流大数据处理的成本。我们首先提出了一个基于8个经过精心设计和验证的转换规则的成本意识工作流转换框架，以降低成本以适应SW的连续执行特性。然后，我们提出联合的VM交通优化问题，并证明它是NP难的。为了在多项式时间内产生最优解，我们同时考虑流量和虚拟机价格异质性，将软件分配问题转换为最小成本最大流量问题。最后，我们的实验结果通过优化工作流程规范和联合VM交通成本优化，以较低的计算和通信成本验证了我们方法的高成本效率。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号