• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Journal of applied measurement >Using Repeated Ratings to Improve Measurement Precision in Incomplete Rating Designs
【24h】

Using Repeated Ratings to Improve Measurement Precision in Incomplete Rating Designs

机译：在不完整的评估设计中使用重复评估来提高测量精度

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

When selecting a design for rater-mediated assessments, one important consideration is the number of raters who rate each examinee. In balancing costs and rater-coverage, rating designs are often implemented wherein only a portion of the examinees are rated by each judge, resulting in large amounts of missing data. One drawback to these sparse rating designs is the reduced precision of examinee ability estimates they provide. When increasing the number of raters per examinee is not feasible, another option may be to increase the number of ratings provided by each rater per examinee. This study applies a Rasch model to explore the effect of increasing the number of rating occasions used by raters to judge examinee proficiency. We used a simulation study to approximate a sparse but connected rater network with a sequentially increasing number of repeated ratings per examinee. The generated data were used to explore the influence of repeated ratings on the precision of rater, examinee, and task parameter estimates as measured by parameter standard errors, the correlation of sparse parameter estimates to true estimates, and the root mean square error of parameter estimates. Results suggest that increasing the number of rating occasions significantly improves the precision of examinee and rater parameter estimates. Results also suggest that parameter recovery levels of rater and task estimates are quite robust to reductions in the number of repeated ratings, although examinee parameter estimates are more sensitive to them. Implications for research and practice in the context of rater-mediated assessment designs are discussed.
机译：在选择评估者介导的评估设计时，一个重要的考虑因素是对每位考生进行评估的评估者数量。在平衡成本和评估者覆盖率时，通常采用评估设计，其中每个法官仅对一部分应试者进行评估，从而导致大量数据丢失。这些稀疏评级设计的缺点是它们提供的考生能力估计的精度降低。当增加每个考生的评分者数量不可行时，另一种选择可能是增加每个考生的每个评分者提供的评分数量。这项研究应用Rasch模型来探索增加评分者用来判断应试者熟练程度的评分机会的影响。我们使用模拟研究来估算稀疏但连接的评估者网络，每个受考者的重复评估次数依次增加。生成的数据用于探讨重复评估对评估者，考生和任务参数估算值的精度的影响，该精度由参数标准误差，稀疏参数估算值与真实估算值的相关性以及参数估算值的均方根误差来衡量。结果表明，增加评分机会的数量显着提高了考生和评分者参数估计的准确性。结果还表明，评估者和任务估计的参数恢复级别对于减少重复评级的数量非常有力，尽管考生参数估计对他们更敏感。讨论了评估者介导的评估设计对研究和实践的影响。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号