• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Social Network, Communication and Education >Hadoop Performance Tuning based on Parameter Optimization
【24h】

Hadoop Performance Tuning based on Parameter Optimization

机译：基于参数优化的Hadoop性能调整

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

In order to better verify that Hadoop performance can be improved through optimization of parameters, we can use the following test methods: benchmarking, stability testing, high availability testing, scalability testing, and security testing. In this paper, the benchmark test method is used to verify the optimization of parameters and to optimize the performance of Hadoop. This article mainly focuses on the 17 parameters in Tab.1. The optimization results are shown in Tab.3. The optimization of the parameters was verified by the execution time of the TeraSort algorithm in the benchmark test. During the experiment, the CPU and memory utilization rate, disk IO and network IO throughput and other indicators were collected. Fig.1-3 fully illustrates the comparison between Hadoop and TeraSort algorithm after parameter default value and parameter adjustment. The experimental results show that after the Hadoop parameters are adjusted and optimized, the Hadoop performance tuning is achieved under certain conditions.
机译：为了更好地通过优化参数来改进Hadoop性能，我们可以使用以下测试方法：基准测试，稳定性测试，高可用性测试，可扩展性测试和安全测试。在本文中，基准测试方法用于验证参数的优化并优化Hadoop的性能。本文主要关注Tab.1中的17个参数。优化结果如图3所示。通过基准测试中的Terasort算法的执行时间验证了参数的优化。在实验期间，收集CPU和内存利用率，磁盘IO和网络IO吞吐量和其他指标。图1-3完全说明了参数默认值和参数调整后Hadoop和Terasort算法之间的比较。实验结果表明，在调整Hadoop参数并优化后，在某些条件下实现了Hadoop性能调谐。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号