关键词: 生成语言质量评价, 机器翻译, 自动文摘, 对话系统, 图像标题生成, 故事生成

Abstract: The quality evaluation of machine generated texts largely affects the research of Natural Language Generation (NLG), and has become a bottleneck restricting the development of the field. This paper reviews on the quality evaluation of various NLG tasks in a broad sense including machine translation, automatic summarization, dialogue, image captioning and machine writing with thorough summarization. The paper introduces the features, pros and cons of human evaluation and automatic metrics respectively as well as some open evaluation resources. This review analyzes the different perspective and applications of various evaluation tasks.  The comparative analysis of different evaluation methods can provide reference for method fusion and exploration of key issues. Overall, the quality evaluation of machine-generated language is still limited to the superficial comparison of linguistic forms, and there are many challenges in deeper evaluation at the level of semantic and coherence or cohesion. Based on the analysis of difficulties and current developments, the paper proposes the research tendencies of quality evaluation of generated texts.

Key words: quality evaluation of generated text, machine translation, automatic summarization, dialogue system, image captioning, storytelling