基于无监督预训练的跨语言AMR解析

摘要/Abstract

摘要： 抽象语义表示AMR是将给定文本的语义特征抽象成一个单根的有向无环图。由于缺乏非英文语言的AMR数据集，跨语言AMR解析通常指给定非英文目标语言文本，构建其英文翻译对应的AMR图。目前跨语言AMR解析的相关工作均基于大规模英文-目标语言平行语料或高性能英文-目标语言翻译模型，通过构建(英文，目标语言和AMR)三元平行语料进行目标语言的AMR解析。与该假设不同的是，本文探索在仅具备大规模单语英文和单语目标语言语料的情况下，实现跨语言AMR解析。为此，提出基于无监督预训练的跨语言AMR解析方法。具体地，在预训练过程中，融合无监督神经机器翻译任务、英文和目标语言AMR解析任务;在微调过程中，使用基于英文AMR 2.0转换的目标语言AMR数据集进行单任务微调。基于AMR 2.0和多语言AMR测试集的实验结果表明，所提方法在德文、西班牙文和意大利文上分别获得了67.89%, 68.04%和67.99%的Smatch F1值。

关键词: 跨语言AMR语义解析, 序列到序列模型, 预训练模型

Abstract: AMR (Abstract Meaning Representation) abstracts the semantic features of a given text into a single-root directed acyclic graph. Due to the lack of non-English language AMR datasets, cross-lingual AMR parsing aims to parse non-English text into the corresponding AMR graph of its English translation. Current cross-lingual AMR parsing methods rely on large-scale English-target language parallel corpora or high-performance English-target language translation models to build (English, target language, AMR) triplet parallel corpora for target language AMR parsing. In contrast to this assumption, this paper explores the possibility of achieving cross-lingual AMR parsing with only large-scale monolingual English and target language corpora. To this end, we propose cross-lingual AMR parsing based on unsupervised pretraining. Specifically, during pretraining, we integrate unsupervised neural machine translation tasks, English AMR parsing tasks, and target language AMR parsing tasks. During fine-tuning, we use an English AMR2.0-based target language AMR dataset for single-task fine-tuning. Experimental results on AMR2.0 and a multilingual AMR test set show that our method achieves Smatch F1 scores of 67.89, 68.04, and 67.99 in German, Spanish, and Italian, respectively.

Key words: cross-lingual AMR parsing, seq2seq model, pre-trained model

范林雨, 李军辉, 孔芳. 基于无监督预训练的跨语言AMR解析[J]. 计算机工程与科学, 2024, 46(01): 170-178.

FAN Lin-yu, LI Jun-hui, KONG Fang. Cross-lingual AMR parsing based on unsupervised pre-training[J]. Computer Engineering & Science, 2024, 46(01): 170-178.

参考文献［31］

［1］	Banarescu L,Bonial C,Cai S,et al.Abstract meaning representation for sembanking［C］∥Proc of the 7th Linguistic Annotation Workshop and Interoperability with Discourse,2013:178-186.
［2］	Liao K,Lebanoff L,Liu F.Abstract meaning representation for multi-document summarization［C］∥Proc of the 27th International Conference on Computational Linguistics,2018:1178-1190.
［3］	Song L,Gildea D,Zhang Y,et al.Semantic neural machine translation using AMR［J］.Transactions of the Association for Computational Linguistics,2019,7:19-31.
［4］	Konstas I,Iyer S,Yatskar M,et al.Neural AMR:Sequence-to-sequence models for parsing and generation［C］∥Proc of the 55th Annual Meeting of the Association for Computational Linguistics,2017:146-157.
［5］	Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need［C］∥Proc of the 2008 International Conference and Workshop on Neural Information Processing Systems,2017:5998-6008.
［6］	Ge D,Li J,Zhu M,et al.Modeling source syntax and semantics for neural AMR parsing［C］∥Proc of the 28th International Joint Conference on Artificial Intelligence,2019:4975-4981.
［7］	Xu D, Li J, Zhu M, et al.Improving AMR parsing with sequence-to-sequence pre-training［C］∥Proc of the 2020 Conference on Empirical Methods in Natural Language Processing,2020:2501-2511.
［8］	Yu C,Gildea D.Sequence-to-sequence AMR parsing with ancestor information［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:571-577.
［9］	Bevilacqua M, Blloshmi R,Navigli R.One SPRING to rule them both:Symmetric AMR semantic parsing and generation without a complex pipeline［C］∥Proc of the AAAI Confe- rence on Artificial Intelligence,2021:12564-12573.
［10］	Damonte M,Cohen S B.Cross-lingual abstract meaning representation parsing［C］∥Proc of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics,2018:1146-1155.
［11］	Xu D, Li J,Zhu M,et al.XLPT-AMR:Cross-lingual pre-training via multi-task learning for zero-shot AMR parsing and text generation［C］∥Proc of the 59th Annual Meeting of the Association for Computational Linguistics,2020:896-907.
［12］	Cai Y,Lin Z,Wan X.Making better use of bilingual information for cross-lingual AMR parsing［C］∥Proc of Findings of the Association for Computational Linguistics,2021:1537-1547.
［13］	Lample G, Conneau A.Cross-lingual language model pretraining［C］∥Proc of International Conference on Neural Information Processing Systems,2019:1631-1642.
［14］	Flanigan J,Thomson S,Carbonell J,et al.A discriminative graph-based parser for the abstract meaning representation［C］∥Proc of the 52nd Annual Meeting of the Association for Computational Linguistics,2014:1426-1436.
［15］	Lyu C,Titov I.AMR parsing as graph prediction with latent alignment［C］∥Proc of the 56th Annual Meeting of the Association for Computational Linguistics,2018:397-407.
［16］	Zhang S,Ma X,Duh K,et al.AMR parsing as sequence to graph transduction［C］∥Proc of the 57th Annual Meeting of the Association for Computational Linguistics,2019:80-94.
［17］	Zhou Q, Zhang Y,Ji D,et al.AMR parsing with latent structural information［C］∥Proc of the 58th Annual Meeting of the Association for Computational Linguistics,2020:4306-4319.
［18］	Damonte M,Cohen S B,Satta G.An incremental parser for abstract meaning representation［C］∥Proc of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2017:536-546.
［19］	Guo Z,Lu W.Better transition based AMR parsing with a refined search space［C］∥Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:1712-1722.
［20］	Zhou J,Naseem T,Astudillo R F,et al.AMR parsing with action-pointer transformer［C］∥Proc of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics,2021:5585-5598.
［21］	Zhang S,Ma X T,Duh K,et al.Broad-coverage semantic parsing as transduction［C］∥Proc of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:3777-3789.
［22］	Cai D,Lam W.Core semantic fifirst:A top-down approach for AMR parsing［C］∥Proc of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:3790-3800.
［23］	Konstas I, Iyer S,Yatskar M, et al. Neural AMR: Sequence-to-sequence models for parsing and generation［C］∥Proc of the 55th Annual Meeting of the Association for Computational Linguistics,2017:146-157.
［24］	Noord R V,Bos J.Neural semantic parsing by character-based translation:Experiments with abstract meaning representations［J］.Computational Linguistics in the Netherlands Journal,2017,5:93-108.
［25］	Bai X,Chen Y,Zhang Y.Graph pre-training for AMR parsing and generation［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:6001-6015.
［26］	Vanderwende L, Menezes A,Quirk C.An ARM parser for English,French,German,Spanish and Japanese and a new AMR-annotated corpus［C］∥Proc of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Demonstrations,2015:26-30.
［27］	Lample G,Conneau A,Denoyer L,et al.Unsupervised machine translation using monolingual corpora only［C］∥Proc of International Conference on Learning Representations,2018:94-106.
［28］	Artetxe M,Labaka G,Agirre E,et al.Unsupervised neural machine translation［C］∥Proc of International Conference on Learning Representations,2018:154-167.
［29］	Lample G,Ott M,Conneau A,et al.Phrase-based & neural unsupervised machine translation［C］∥Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:5039-5049.
［30］	Cai S,Knight K.Smatch:An evaluation metric for semantic feature structure［C］∥Proc of the 51st Annual Meeting of the Association for Computational Linguistics,2013:748-752.
［31］	Damonte M,Cohen S B,Satta G.An incremental parser for abstract meaning representation［C］∥Proc of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2016:536-546.