• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Data & Knowledge Engineering >A probabilistic evaluation procedure for process model matching techniques
【24h】

A probabilistic evaluation procedure for process model matching techniques

机译：过程模型匹配技术的概率评估程序

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Process model matching refers to the automatic identification of corresponding activities between two process models. It represents the basis for many advanced process model analysis techniques such as the identification of similar process parts or process model search. A central problem is how to evaluate the performance of process model matching techniques. Current evaluation methods require a binary gold standard that clearly defines which correspondences are correct. The problem is that often not even humans can agree on a set of correct correspondences. Hence, evaluating the performance of matching techniques based on a binary gold standard does not take the true complexity of the matching problem into account and does not fairly assess the capabilities of a matching technique. In this paper, we propose a novel evaluation procedure for process model matching techniques. In particular, we build on the assessments of multiple annotators to define the notion of a non-binary gold standard. In this way, we avoid the problem of agreeing on a single set of correct correspondences. Based on this non-binary gold standard, we introduce probabilistic versions of precision, recall, and F-measure as well as a distance-based performance measure. We use a dataset from the Process Model Matching Contest 2015 and a total of 16 matching systems to assess and compare the insights that can be obtained by using our evaluation procedure. We find that our probabilistic evaluation procedure allows us to gain more detailed insights into the performance of matching systems than a traditional evaluation based on a binary gold standard.
机译：过程模型匹配是指自动识别两个过程模型之间的相应活动。它代表了许多高级过程模型分析技术的基础，例如类似过程零件的标识或过程模型搜索。一个中心问题是如何评估过程模型匹配技术的性能。当前的评估方法需要二进制金标准，该标准明确定义了哪些对应是正确的。问题在于，甚至连人类也常常无法同意一套正确的对应关系。因此，评估基于二进制金标准的匹配技术的性能并没有考虑到匹配问题的真正复杂性，也没有公平地评估匹配技术的能力。在本文中，我们提出了一种新的过程模型匹配技术评估程序。特别是，我们基于对多个注释器的评估来定义非二进制黄金标准的概念。这样，我们避免了在一组正确的对应关系上达成一致的问题。基于此非二进制金标准，我们介绍了精度，查全率和F度量以及基于距离的性能度量的概率版本。我们使用来自2015年流程模型匹配竞赛的数据集和总共16个匹配系统来评估和比较通过使用我们的评估程序可以获得的见解。我们发现，与基于二进制黄金标准的传统评估相比，我们的概率评估程序使我们能够更深入地了解匹配系统的性能。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号