• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Reliability >Minimum Backups for Stream Processing With Recovery Latency Guarantees
【24h】

Minimum Backups for Stream Processing With Recovery Latency Guarantees

机译：具有恢复延迟保证的流处理的最小备份

获取原文
获取原文并翻译 | 示例
       
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The stream processing model continuously processes online data in an on-pass fashion that can be more vulnerable to failures than other big-data processing schemes. Existing fault-tolerant (FT) approaches have been presented to enhance the reliability of stream processing systems. However, the fundamental tradeoff between recovery latency and FT overhead is still unclear, so these scheme cannot provide recovery latency guarantees. This paper introduces the FT Configuration (FTC) problem and presents a solution for guaranteed recovery latency with minimum backups. A failure effect model is presented to describe the relationship between recovery latency and FTC (the amount and locations of backups). With this model, we design an algorithm to compute FTCs for different types of stream topologies according to recovery latency requirements. Extensive experiments are conducted to verify the correctness and effectiveness of our approach. We prove that our algorithm guarantees recovery latencies for all directed acyclic graph (DAG) stream topologies. For line(s) and tree topologies, our algorithm solves the FTC problem with a time complexity of O(N). For a general DAG topology, a heuristic function is used to generate FTCs. This causes fewer than 10% more backups on average compared to the optimal solution with a time complexity of O(N).
机译：流处理模型以直通方式连续处理在线数据，这种方式比其他大数据处理方案更容易发生故障。已经提出了现有的容错（FT）方法来增强流处理系统的可靠性。但是，恢复延迟和FT开销之间的基本权衡仍然不清楚，因此这些方案无法提供恢复延迟保证。本文介绍了FT配置（FTC）问题，并提出了一种以最少的备份保证恢复延迟的解决方案。提出了一个故障影响模型来描述恢复延迟和FTC（备份的数量和位置）之间的关系。使用此模型，我们设计了一种算法，可根据恢复延迟要求为不同类型的流拓扑计算FTC。进行了广泛的实验以验证我们方法的正确性和有效性。我们证明了我们的算法可以保证所有有向无环图（DAG）流拓扑的恢复延迟。对于线和树拓扑，我们的算法以O（N）的时间复杂度解决了FTC问题。对于一般的DAG拓扑，启发式函数用于生成FTC。与时间复杂度为O（N）的最佳解决方案相比，这平均会使备份增加不到10％。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号