• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Malaysian Journal of Computer Science >Relevance Judgments Exclusive of Human Assessors in Large Scale Information Retrieval Evaluation Experimentation
【24h】

Relevance Judgments Exclusive of Human Assessors in Large Scale Information Retrieval Evaluation Experimentation

机译：大规模信息检索评估实验中不包含人类评估者的相关性判断

获取原文
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Inconsistent judgments by various human assessors?compromises the reliability of the relevance judgments generated for large scale test collections. An automated method that creates a similar set of relevance judgments (pseudo relevance judgments) that eliminate the human efforts and errors introduced in creating relevance judgments is investigated in this study. Traditionally, the participating systems in TREC are measured by using a chosen metrics and ranked according to its performance scores. In order to generate these scores, the documents retrieved by these systems for each topic are matched with the set of relevance judgments (often assessed by humans). In this study, the number of occurrences of each document per topic from the various runs will be used with an assumption, the higher the number of occurrences of a document, the possibility of the document being relevant is higher. The study proposesa method with a pool depth of 100 using the cutoff percentage of >35% that could provide an alternate way of generating consistent relevance judgments without the involvement of human assessors.
机译：各种评估人员的判断不一致会损害为大型测试集合生成的相关判断的可靠性。在这项研究中，研究了一种自动方法，该方法创建了一组相似的相关性判断（伪相关性判断），从而消除了在创建相关性判断时引入的人工和错误。传统上，TREC中的参与系统是通过使用选定的指标进行衡量的，并根据其性能得分进行排名。为了生成这些分数，将这些系统为每个主题检索的文档与相关性判断集匹配（通常由人员进行评估）。在这项研究中，每个主题中每个主题的每个文档的出现次数将与假设一起使用，文档的出现次数越高，文档相关的可能性就越高。该研究提出了一种使用100％池深度的方法，该方法使用大于35％的截断百分比，这可以提供另一种方式来生成一致的相关性判断，而无需人工评估。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号