...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Expert systems: The international journal of knowledge engineering >How interesting and coherent are the stories generated by a large-scale neural language model? Comparing human and automatic evaluations of machine-generated text
【24h】

How interesting and coherent are the stories generated by a large-scale neural language model? Comparing human and automatic evaluations of machine-generated text

机译：How interesting and coherent are the stories generated by a large-scale neural language model? Comparing human and automatic evaluations of machine-generated text

获取原文
获取原文并翻译 | 示例
   

获取外文期刊封面封底 >>

       
页面导航
  • 摘要
  • 著录项
  • 相关主题

摘要

Evaluation of the narrative text generated by machines has traditionally been a challenge,particularly when attempting to evaluate subjective elements such as interestor believability. Recent improvements in narrative machine text generation havebeen largely driven by the emergence of transformer-based language models, trainedon massive quantities of data, resulting in higher quality text generation. In this study,a corpus of stories is generated using the pre-trained GPT-Neo transformer model,with human-written prompts as inputs upon which to base the narrative text. Thestories generated through this process are subsequently evaluated through bothhuman evaluation and two automated metrics: BERTScore and BERT Next SentencePrediction, with the aim of determining whether there is a correlation between theautomatic scores and the human judgements. The results show variation in humanevaluation results in comparison to modern automated metrics, suggesting furtherwork is required to train automated metrics to identify text that is defined as interestingby humans.

著录项

获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号