• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International conference on world wide web >Contextual and Dimensional Relevance Judgments for Reusable SERP-Level Evaluation
【24h】

Contextual and Dimensional Relevance Judgments for Reusable SERP-Level Evaluation

机译：可重复使用SERP级评估的上下文和维度相关性判断

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Document-level relevance judgments are a major component in the calculation of effectiveness metrics. Collecting high-quality judgments is therefore a critical step in information retrieval evaluation. However, the nature of, and the assumptions underlying, relevance judgment collection have not received much attention. In particular, relevance judgments are typically collected for each document in isolation, although users read each document in the context of other documents. In this work, we aim to investigate the nature of relevance judgment collection. We collect relevance labels in both isolated and conditional settings, and ask for judgments in various dimensions of relevance, as well as overall relevance. Then we compare the relevance metrics based on various types of judgments with other metrics of quality such as User Preference. Our analyses illuminate how these settings for judgment collection affect the quality and the characteristics of the judgments. We also find that the metrics based on conditional judgments show higher correlation with user preference than isolated judgments.
机译：文档级别的相关性判断是有效性指标计算中的主要组成部分。因此，收集高质量的判断是信息检索评估中的关键步骤。但是，相关性判断收集的性质和基础假设并没有引起足够的重视。特别是，尽管用户是在其他文档的上下文中阅读每个文档的，但是通常会单独为每个文档收集相关性判断。在这项工作中，我们旨在调查相关性判断收集的性质。我们在孤立和有条件的环境中收集相关性标签，并要求对相关性的各个方面以及整体相关性做出判断。然后，我们将基于各种类型判断的相关性指标与其他质量指标（例如用户偏好）进行比较。我们的分析阐明了这些用于判断收集的设置如何影响判断的质量和特征。我们还发现，基于条件判断的指标与孤立偏好相比，与用户偏好之间的相关性更高。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号