• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Applied Measurement in Education >Validating human and automated scoring of essays against 'True' scores
【24h】

Validating human and automated scoring of essays against 'True' scores

机译：验证散文的人类和自动评分对“真实”分数

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

In the current study, two pools of 250 essays, all written as a response to the same prompt, were rated by two groups of raters (14 or 15 raters per group), thereby providing an approximation to the essay's true score. An automated essay scoring (AES) system was trained on the datasets and then scored the essays using a cross-validation scheme. By eliminating one, two, or three raters at a time, and by calculating an estimate of the true scores using the remaining raters, an independent criterion against which to judge the validity of the human raters and that of the AES system, as well as the interrater reliability was produced. The results of the study indicated that the automated scores correlate with human scores to the same degree as human raters correlate with each other. However, the findings regarding the validity of the ratings support a claim that the reliability and validity of AES diverge: although the AES scoring is, naturally, more consistent than the human ratings, it is less valid.
机译：在目前的研究中，两个250篇论文的池是由两组评价者（每组14或15个评估者）评定的两组课程，从而为论文的真实得分提供近似值。在数据集上培训自动化论文评分（AES）系统，然后使用交叉验证方案对散文进行评分。通过一次消除一个，两个或三个评估者，并且通过使用剩余的评估者计算真实分数的估计，这是一个独立的标准，用于判断人类评估者的有效性以及AES系统的有效性，以及产生的Interray可靠性。该研究的结果表明，随着人类评估者彼此相关的，自动分数与人类评分相关的程度相同。然而，关于评级的有效性的调查结果支持AES分歧的可靠性和有效性：虽然AES得分是自然的，但比人类评分更加一致，但它不太有效。

著录项

  • 来源
    《Applied Measurement in Education》 |2018年第3期|共10页
  • 作者

    Cohen Yoav; Levi Effi; Ben-Simon Anat;

  • 作者单位

    Natl Inst Testing &

    Evaluat POB 26015 IL-9126001 Jerusalem Israel;

    Natl Inst Testing &

    Evaluat POB 26015 IL-9126001 Jerusalem Israel;

    Natl Inst Testing &

    Evaluat POB 26015 IL-9126001 Jerusalem Israel;

  • 收录信息
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类 计量学;
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号