一个面向信息抽取的中英文平行语料库

J4 ›› 2015, Vol. 37 ›› Issue (12): 2331-2338.

一个面向信息抽取的中英文平行语料库

惠浩添，李云建，钱龙华，周国栋

(1.苏州大学自然语言处理实验室，江苏苏州 215006；2.苏州大学计算机科学与技术学院，江苏苏州 215006)

收稿日期:2015-08-26 修回日期:2015-10-21 出版日期:2015-12-25 发布日期:2015-12-25
基金资助:
国家自然科学基金资助项目(61373096，90920004)；江苏省高校自然科学研究重大项目(11KJA520003)

A ChineseEnglish parallel corpus for information extraction

HUI Haotian，LI Yunjian，QIAN Longhua，ZHOU Guodong

(1.Natural Language Processing Lab,Soochow University,Suzhou 215006;2.School of Computer Science & Technology,Soochow University,Suzhou 215006,China)

Received:2015-08-26 Revised:2015-10-21 Online:2015-12-25 Published:2015-12-25

摘要/Abstract

摘要：

除了机器翻译，平行语料库对信息检索、信息抽取及知识获取等研究领域具有重要的作用，但是传统的平行语料库只是在句子级对齐，因而对跨语言自然语言处理研究的作用有限。鉴于此，以OntoNotes中英文平行语料库为基础，通过自动抽取、自动映射加人工标注相结合的方法，构建了一个面向信息抽取的高质量中英文平行语料库。该语料库不仅包含中英文实体及其相互关系，而且实现了中英文在实体和关系级别上的对齐。因此，该语料库将有助于中英文信息抽取的对比研究，揭示不同语言在语义表达上的差异，也为跨语言信息抽取的研究提供了一个有价值的平台。

关键词: 命名实体, 语义关系, 双语映射, 平行语料库

Abstract:

In addition to machine translation, parallel corpora play an important role in information retrieval, information extraction and knowledge acquisition, etc. However, traditional parallel corpora are aligned at sentence level, thus their significance for research on crosslanguage natural language processing is limited. In view of this, on the basis of the OntoNotes, we construct a high quality Chinese and English parallel corpus for information extraction by combining automatic extraction, automatic mapping and manual annotation. The corpus contains the entities and their mutual relations, and achieves the alignment between Chinese and English both on entity and relation levels. This corpus therefore can facilitate comparative study of information extraction in Chinese and English, reveal the difference of semantic expressions between languages, and also provide a valuable platform for research on cross-language information extraction.Key words:

Key words: named entity;semantic relation;bilingual mapping;parallel corpus

惠浩添，李云建，钱龙华，周国栋. 一个面向信息抽取的中英文平行语料库[J]. J4, 2015, 37(12): 2331-2338.

HUI Haotian，LI Yunjian，QIAN Longhua，ZHOU Guodong. A ChineseEnglish parallel corpus for information extraction [J]. J4, 2015, 37(12): 2331-2338.

参考文献

［1］Doddington G R,Mitchell A,Przybocki M A,et al.The automatic content extraction （ACE） programtasks,data,and evaluation［C］∥Proc of LREC, 2004:837840．
［2］Chen Y,Zong C,Su K Y.On jointly recognizing and aligning bilingual named entities［C］∥Proc of the 48th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2010:631639．
［3］Kim S,Jeong M,Lee J,et al.Crosslingual annotation projection for weaklysupervised relation extraction［J］.ACM Transactions on Asian Language Information Processing （TALIP）,2014,13（1）:126．
［4］Qian L,Hui H,Hu Y,et al.Bilingual active learning for relation classification via pseudo parallel corpora［C］∥Proc of ACL, 2014:582592．
［5］Xiao R.The babel EnglishChinese parallel corpus［DB/OL］.［20130213］.http:∥www.lancaster.ac.ukfassprojects/corpus/babel/babel.htm．
［6］Ma Xiaoyi.Hong Kong parallel text LDC2004T08［R］.Philadelphia:Linguistic Data Consortium,2004．
［7］United States.Joint Publications Research Service,United States.Foreign Broadcast Information Service.JPRS Report:China［M］.Washington DC:Foreign Broadcast Information Service,1993．
［8］Deng Y,Kumar S,Byrne W.Segmentation and alignment of parallel text for statistical machine translation［J］.Natural Language Engineering,2007,13（3）:235260．
［9］Ma X.Champollion:A robust parallel text sentence aligner［C］∥LREC 2006:Fifth International Conference on Language Resources and Evaluation, 2006:489492．
［10］Li P, Sun M, Xue P. FastChampollion:A fast and robust sentence alignment algorithm［C］∥Proc of the 23rd International Conference on Computational Linguistics:Posters.Association for Computational Linguistics,2010:710718．
［11］Quan X,Kit C,Song Y.Nonmonotonic sentence alignment via semisupervised learning［C］∥Proc of ACL, 2013:622630．
［12］Brown P F,Pietra V J D,Pietra S A D,et al.The mathematics of statistical machine translation:Parameter estimation［J］.Computational Linguistics,1993,19（2）:263311．
［13］Vogel S,Ney H,Tillmann C.HMMbased word alignment in statistical translation［C］∥Proc of the 16th Conference on Computational Linguistics,1996:836841．
［14］Feng D,Lü Y,Zhou M.A new approach for EnglishChinese named entity alignment［C］∥Proc of EMNLP’04,2004:372379．

[1]	喻金平, 朱伟锋, 廖列法. 基于RoBERTa-wwm-BiLSTM-CRF的扶持政策文本实体识别研究[J]. 计算机工程与科学, 2023, 45(08): 1498-1507.
[2]	李鸿飞, 刘盼雨, 魏勇. 基于自注意力和Lattice-LSTM的军事命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1848-1855.
[3]	许力, 李建华. 基于BERT和BiLSTM-CRF的生物医学命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1873-1879.
[4]	仇增辉, 赫明杰, 林正奎. 基于深度学习的网购评论命名实体识别方法[J]. 计算机工程与科学, 2020, 42(12): 2287-2294.
[5]	肖新凤1,2，李石君2，余伟2，刘杰2，刘倍雄1. 基于改进seq2seq模型的英汉翻译研究[J]. 计算机工程与科学, 2019, 41(07): 1257-1265.
[6]	李健龙，王盼卿，韩琪羽. 基于双向LSTM的军事命名实体识别[J]. 计算机工程与科学, 2019, 41(04): 711-718.
[7]	张朝胜1，郭剑毅1,2，线岩团1,2，余正涛1,2，雷春雅1, 王海雄1. 基于条件随机场的英文产品命名实体识别[J]. J4, 2010, 32(6): 115-117.
[8]	路松峰王丹丹. 基于网页格局的内容分块算法[J]. J4, 2007, 29(9): 16-18.
[9]	施德明林洋港陈恩红. 一种集成NER的文本分类特征选择方法[J]. J4, 2007, 29(11): 152-156.
[10]	张晓艳王挺陈火旺. 基于混合统计模型的汉语命名实体识别方法[J]. J4, 2006, 28(6): 135-139.