• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文学位 >Metamorphic testing techniques to detect defects in applications without test oracles.
【24h】

Metamorphic testing techniques to detect defects in applications without test oracles.

机译：无需测试先兆即可检测应用程序中的缺陷的变形测试技术。

获取原文
获取原文并翻译 | 示例
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Applications in the fields of scientific computing, simulation, optimization, machine learning, etc. are sometimes said to be "non-testable programs" because there is no reliable test oracle to indicate what the correct output should be for arbitrary input. In some cases, it may be impossible to know the program's correct output a priori; in other cases, the creation of an oracle may simply be too hard. These applications typically fall into a category of software that Weyuker describes as "Programs which were written in order to determine the answer in the first place. There would be no need to write such programs, if the correct answer were known." The absence of a test oracle clearly presents a challenge when it comes to detecting subtle errors, faults, defects or anomalies in software in these domains.;As these types of programs become more and more prevalent in various aspects of everyday life, the dependability of software in these domains takes on increasing importance. Machine learning and scientific computing software may be used for critical tasks such as helping doctors perform a medical diagnosis or enabling weather forecasters to more accurately predict the paths of hurricanes; hospitals may use simulation software to understand the impact of resource allocation on the time patients spend in the emergency room. Clearly, a software defect in any of these domains can cause great inconvenience or even physical harm if not detected in a timely manner.;Without a test oracle, it is impossible to know in general what the expected output should be for a given input, but it may be possible to predict how changes to the input should effect changes in the output, and thus identify expected relations among a set of inputs and among the set of their respective outputs. This approach, introduced by Chen et al., is known as "metamorphic testing". In metamorphic testing, if test case input x produces an output f(x), the function's so-called "metamorphic properties" can then be used to guide the creation of a transformation function t, which can then be applied to the input to produce t(x); this transformation then allows us to predict the expected output f(t(x)), based on the (already known) value of f(x). If the new output is as expected, it is not necessarily right, but any violation of the property indicates that one (or both) of the outputs is wrong. That is, though it may not be possible to know whether an output is correct, we can at least tell whether an output is incorrect.;This thesis investigates three hypotheses. First, I claim that an automated approach to metamorphic testing will advance the state of the art in detecting defects in programs without test oracles, particularly in the domains of machine learning, simulation, and optimization. To demonstrate this, I describe a tool for test automation, and present the results of new empirical studies comparing the effectiveness of metamorphic testing to that of other techniques for testing applications that do not have an oracle. Second, I claim that conducting function-level metamorphic testing in the context of a running application will reveal defects not found by metamorphic testing using system-level properties alone, and introduce and evaluate a new testing technique called Metamorphic Runtime Checking. Third, I hypothesize that it is feasible to continue this type of testing in the deployment environment (i.e., after the software is released), with minimal impact on the user, and describe an approach called In Vivo Testing.;Additionally, this thesis presents guidelines for identifying metamorphic properties, explains how metamorphic testing fits into the software development process, and discusses suggestions for both practitioners and researchers who need to test software without the help of a test oracle.
机译：有时将在科学计算，模拟，优化，机器学习等领域中的应用程序称为“不可测试的程序”，因为没有可靠的测试指示符来指示任意输入的正确输出。在某些情况下，可能无法先验地知道程序的正确输出。在其他情况下，创建甲骨文可能太难了。这些应用程序通常属于Weyuker所描述的一种软件类别，即“最初为了确定答案而编写的程序。如果知道正确的答案，则无需编写此类程序。”在这些领域中检测软件中的细微错误，错误，缺陷或异常时，缺少测试oracle显然提出了挑战。随着这些类型的程序在日常生活的各个方面变得越来越普遍，这些领域的软件变得越来越重要。机器学习和科学计算软件可用于关键任务，例如帮助医生进行医学诊断或使天气预报员更准确地预测飓风的路径；医院可以使用模拟软件来了解资源分配对患者在急诊室花费时间的影响。显然，如果不及时检测，这些领域中的任何一个软件缺陷都可能造成极大的不便甚至是人身伤害。;如果没有测试预告片，通常就无法知道给定输入的预期输出是什么，但是可以预测输入的更改应如何影响输出中的更改，从而确定一组输入之间以及它们各自的输出之间的预期关系。 Chen等人介绍的这种方法称为“变形测试”。在变形测试中，如果测试用例输入x产生输出f（x），则可以使用函数的所谓“变形特性”来指导转换函数t的创建，然后可以将其应用于输入以产生t（x）;然后，此转换使我们能够基于f（x）的（已知）值来预测预期输出f（t（x））。如果新输出符合预期，则不一定正确，但是任何违反此属性的行为都表明一个（或两个）输出错误。也就是说，尽管可能无法知道输出是否正确，但我们至少可以判断输出是否不正确。;本文研究了三个假设。首先，我声称一种自动化的变态测试方法将在检测没有测试预言的程序中的缺陷方面，特别是在机器学习，仿真和优化领域中，提高检测水平。为了证明这一点，我描述了一种用于测试自动化的工具，并提供了新的实证研究结果，将变质测试的有效性与其他没有Oracle的应用程序的测试技术进行了比较。其次，我声称在正在运行的应用程序上下文中进行功能级别的变形测试将发现仅使用系统级别属性进行的变形测试无法发现的缺陷，并引入并评估了一种称为“变形运行时检查”的新测试技术。第三，我假设在部署环境中（即软件发布后）继续进行这种类型的测试是可行的，并且对用户的影响最小，并描述了一种称为“体内测试”的方法。识别变质性质的准则，解释变质测试如何适合软件开发过程，并为需要在不使用测试Oracle的情况下测试软件的从业人员和研究人员提供建议。

著录项

  • 作者

    Murphy, Christian.;

  • 作者单位

    Columbia University.;

  • 授予单位 Columbia University.;
  • 学科 Engineering Computer.;Computer Science.
  • 学位 Ph.D.
  • 年度 2010
  • 页码 232 p.
  • 总页数 232
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号