• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>IEEE International Conference on Software Maintenance and Evolution >An Exploratory Study of Performance Regression Introducing Code Changes
【24h】

An Exploratory Study of Performance Regression Introducing Code Changes

机译：绩效回归介绍代码改变的探索性研究

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Performance is an important aspect of software quality. In fact, large software systems failures are often due to performance issues rather than functional bugs. One of the most important performance issues is performance regression. Examples of performance regressions are response time degradation and increased resource utilization. Although performance regressions are not all bugs, they often have a direct impact on users' experience of the system. Due to the possible large impact of performance regressions, prior research proposes various automated approaches that detect performance regressions. However, the detection of performance regressions is conducted after the fact, i.e., after the system is built and deployed in the field or dedicated performance testing environments. On the other hand, there exists rich software quality research that examines the impact of code changes on software quality; while a majority of prior findings do not use performance regression as a sign of software quality degradation. In this paper, we perform an exploratory study on the source code changes that introduce performance regressions. We conduct a statistically rigorous performance evaluation on 1,126 commits from ten releases of Hadoop and 135 commits from five releases of RxJava. In particular, we repetitively run tests and performance micro-benchmarks for each commit while measuring response time, CPU usage, Memory usage and I/O traffic. We identify performance regressions in each test or performance micro-benchmark if there exists statistically significant degradation with medium or large effect sizes, in any performance metric. We find that performance regressions widely exist during the development of both subject systems. By manually examining the issue reports that are associated with the identified performance regression introducing commits, we find that the majority of the performance regressions are introduced while fixing other bugs. In addition, we identify six root-causes of performance regressions. 12.5% of the examined performance regressions can be avoided or their impact may be reduced during development. Our findings highlight the need for performance assurance activities during development. Developers should address avoidable performance regressions and be aware of the impact of unavoidable performance regressions.
机译：性能是软件质量的一个重要方面。事实上，大型软件系统故障通常是由于性能问题而不是功能性错误。最重要的性能问题之一是性能回归。性能回归的示例是响应时间劣化和提高资源利用率。虽然性能回归不是所有错误，但它们通常对用户的体验有直接影响。由于性能回归可能的巨大影响，先验研究提出了检测性能回归的各种自动化方法。然而，在事实之后进行性能回归的检测，即在系统中构建并部署在现场或专用性能测试环境之后。另一方面，存在丰富的软件质量研究，检查代码变化对软件质量的影响;虽然大多数现有结果不使用性能回归作为软件质量退化的标志。在本文中，我们对引入性能回归的源代码更改进行探索性研究。我们在从RXJava的五个版本的Hadoop和135次提交的1,126次提交上进行统计严格的绩效评估。特别是，我们重复运行每次提交的测试和性能微基准，同时测量响应时间，CPU使用率，内存使用和I / O流量。如果在任何性能度量中，我们在任何具有统计学或大效果尺寸的情况下存在统计学上显着的降级，则我们识别每个测试或性能微基准的性能回归。我们发现在两个主题系统的开发期间广泛存在的性能回归。通过手动检查与识别的绩效回归引入提交相关联的问题报告，我们发现在修复其他错误的同时引入了大多数性能回归。此外，我们确定了六种性能回归的根本原因。 12.5 ％可以避免检验的性能回归，或者在开发期间可能会降低其影响。我们的研究结果强调了在开发期间对性能保证活动的需求。开发人员应解决可避免的性能回归，并意识到不可避免的性能回归的影响。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号