基于多视角对比学习的隐式篇章关系识别

计算机工程与科学 ›› 2024, Vol. 46 ›› Issue (04): 716-724.

基于多视角对比学习的隐式篇章关系识别

吴一珩1,李军辉1,朱慕华2

（1.苏州大学计算机科学与技术学院，江苏苏州 215006；2.美团,北京 100000）

收稿日期:2022-12-24 修回日期:2023-05-12 接受日期:2024-04-25 出版日期:2024-04-25 发布日期:2024-04-18
基金资助:
国家自然科学基金 (61876120)

Implicit discourse relation recognition with multi-view contrastive learning

WU Yi-heng1,LI Jun-hui1,ZHU Mu-hua2

(1.School of Computer Science & Technology,Soochow University,Suzhou 215006;
2.Meituan,Beijing 100000,China)

Received:2022-12-24 Revised:2023-05-12 Accepted:2024-04-25 Online:2024-04-25 Published:2024-04-18

摘要/Abstract

摘要： 隐式篇章关系识别IDRR的相关工作集中在篇章单元编码器的设计上。将对比学习引入到IDRR，以此获得区分度更高的篇章单元表征。具体地，首先使用一个轻量的IDRR模型；然后为了学习到更好的篇章单元表征，分别从样例层级、批层级和群层级，探索了3种不同视角的对比学习方法在IDRR中的应用；最后本文将多视角对比学习目标联合IDRR同时进行训练。本文提出的方法几乎不增加训练时间，而且只引入少量额外参数。基于PDTB 2.0的实验结果表明该方法达到了最优性能。

关键词: 隐式篇章关系识别, 多视角, 对比学习, 联合学习

Abstract: Previous researches on implicit discourse relationship recognition (IDRR) usually focus on designing effective discourse encoders. Different from theirs, this paper proposes a novel approach which introduces contrastive learning into IDRR so as to obtain representations of discourse units (DUs) with more differentiation. Specifically, a lightweight IDRR classification model is firstly adopted. Then, to better learn representations of DUs, the application of three different contrastive learning methods in IDDR are explored from multiple views, including instance-level, batch-level, and group- level. Finally, three multi-view contrastive learning objectives are combined for better IDRR. Our proposed method only slightly increases training time and introduces small additional parameters. Experimental results on PDTB 2.0 show that our method achieves the state-of-the-art performance.

Key words: implicit discourse relation recognition, multi-view, contrastive learning, joint learning

吴一珩, 李军辉, 朱慕华 . 基于多视角对比学习的隐式篇章关系识别[J]. 计算机工程与科学, 2024, 46(04): 716-724.

WU Yi-heng, LI Jun-hui, ZHU Mu-hua. Implicit discourse relation recognition with multi-view contrastive learning[J]. Computer Engineering & Science, 2024, 46(04): 716-724.

参考文献［32］

［1］	Li J,Carpuat M,Nenkova A.Assessing the discourse factors that influence the quality of machine translation［C］∥Proc of the 52nd Annual Meeting of the Association for Computational Linguistics,2014:283-288.
［2］	Jansen P,Surdeanu M,Clark P.Discourse complements lexical semantics for non-factoid answer re-ranking［C］∥Proc of the 52nd Annual Meeting of the Association for Computational Linguistics,2014:977-986.
［3］	Lin Z H,Kan M-Y,Ng H T.Recognizing implicit discourse relations in the Penn Discourse Treebank［C］∥Proc of the 2009 Conference on Empirical Methods in Natural Language Processing,2009:343-351.
［4］	Pitler E,Louis A,Nenkova A.Automatic sense prediction for implicit discourse relations in text［C］∥Proc of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP,2009:683-691.
［5］	Pitler E,Nenkova A.Using syntax to disambiguate explicit discourse connectives in text［C］∥Proc of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP，2009：13-16.
［6］	Peters M,Neumann M,Iyyer M,et al.Deep contextualized word representations［C］∥Proc of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2018:2227-2237.
［7］	Devlin J,Chang M,Lee K,et al.BERT:Pre-training of deep bidirectional transformers for language understanding［C］∥Proc of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019:4171-4186.
［8］	Liu Y H,Ott M,Goyal N,et al. RoBERTa:A robustly optimized BERT pretraining approach［J］.arXiv:1907.11692,2019.
［9］	Wu C,Cao L,Ge Y,et al.A label dependence-aware sequence generation model for multi-level implicit discourse relation recognition［C］∥Proc of the AAAI Conference on artificial Intelligence,2022:11486-11494.
［10］	Bai H X,Zhao H.Deep enhanced representation for implicit discourse relation recognition［C］∥Proc of the 27th International Conference on Computational Linguistics,2018:571-583.
［11］	Ruan H B,Hong Y,Xu Y,et al.Interactively-propagative attention learning for implicit discourse relation recognition［C］∥Proc of the 28th International Conference on Computational Linguistics,2020:3168-3178.
［12］	Liu X,Ou J,Song Y,et al.On the importance of word and sentence representation learning in implicit discourse relation classification［C］∥Proc of the 29th International Joint Conference on Artificial Intelligence,2020:3830-3836.
［13］	Gao T,Yao X,Chen D.SimCSE:Simple contrastive learning of sentence embeddings［C］∥Proc of the 2021 Conference on Empirical Methods in Natural Language Processing,2021:6894-6910.
［14］	Wang D,Ding N,Li P,et al.CLINE:Contrastive learning with semantic negative examples for natural language understanding［C］∥Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proces- sing,2021:2332-2342.
［15］	Yan Y,Li R,Wang S,et al.ConSERT:A contrastive framework for self-supervised sentence representation transfer［C］∥Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing,2021:5065-5075.
［16］	Zhang Z,Zhao Y,Chen M,et al.Label anchored contrastive learning for language understanding［C］∥Proc of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2022:1437-1449.
［17］	Suresh V,Ong D C.Not all negatives are equal:Label-aware contrastive loss for fine-grained text classification［C］∥Proc of the 2021 Conference on Empirical Methods in Natural Language Processing,2021:4381-4394.
［18］	Pitler E,Raghupathy M,Mehta H,et al.Easily identifiable discourse relations［C］∥Proc of the 22nd International Conference on Computational Linguistics,2008:87-90.
［19］	Dai Z,Huang R.Improving implicit discourse relation classification by modeling interdependencies of discourse units in a paragraph［C］∥Proc of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics,2018:141-151.
［20］	Chen J,Zhang Q,Liu P,et al.Implicit discourse relation detection via a deep architecture with gated relevance network［C］∥Proc of the 54th Annual Meeting of the Association for Computational Linguistics,2016:1726-1735.
［21］	Liu Y,Li S J.Recognizing implicit discourse relations via repeated reading:Neural networks with multi-level attention［C］∥Proc of the 2016 Conference on Empirical Methods in Natural Language Processing,2016:1224-1233.
［22］	He R,Wang J,Guo F,et al.Trans-driven joint learning architecture for implicit discourse relation recognition［C］∥Proc of the 58th Annual Meeting of the Association for Computational Linguistics,2020:139-148.
［23］	孙静，李艳翠，周国栋，等.汉语隐式篇章关系识别［J］.北京大学学报（自然科学版）,2014,50(1):111-117.
	Sun Jing,Li Yan-cui,Zhou Guo-dong,et al.Research of Chinese implicit discourse relation recognition ［J］.Acta Scientiarum Naturalium Universitatis Pekinensis,2014,50(1):111-117.
［24］	徐凡，朱巧明，周国栋.基于树核的隐式篇章关系识别［J］.软件学报,2013,24(5):1022-1035.
	Xu Fan,Zhu Qiao-ming,Zhou Guo-dong.Implicit discourse relation recognition based on tree kernel［J］.Journal of Software,2013,24(5):1022-1035.
［25］	邢雨晴，孔芳.基于多层局部推理的汉语篇章关系及主次联合识别［J］.中文信息学报,2022,36(7):42-49.
	Xing Yu-qing,Kang Fang.Multi-layer local inference based Chinese discourse relation and nuclearity recognition［J］.Journal of Chinese Information Processing,2022,36(7):42-49.
［26］	Gunel B,Du J F,Conneau A,et al.Supervised contrastive learning for pre-trained language model fine-tuning［J］. arXiv:2011.01403,2020.
［27］	Prasad R,Dinesh N,Lee A,et al.The Penn Discourse TreeBank 2.0［C］∥Proc of the 6th International Conference on Language Resources and Evaluation,2008:2961-2968.
［28］	Ji Y,Eisenstein J.One vector is not enough:Entity-augmented distributed semantics for discourse relations［J］.Transactions of the Association for Computational Linguistics,2019,3:329-344.
［29］	Zhang Y,Meng F,Li P,et al.Context tracking network:Graph-based context modeling for implicit discourse relation recognition ［C］∥Proc of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2021:1592-1599.
［30］	Varia S,Hidey C,Chakrabarty T.Discourse relation prediction:Revisiting word pairs with convolutional networks［C］∥Proc of the 20th Annual SGIdial Meeting on Discourse and Dialogue,2019:442-452.
［31］	Wu C X,Hu C W,Li R C,et al.Hierarchical multi-task learning with CRF for implicit discourse relation recognition［J］.Knowledge-Based Systems,2020,195:105637.
［32］	Jiang F,Fan Y,Chu X,et al.Not just classification:Recognizing implicit discourse relation on joint modeling of classification and generation［C］∥Proc of the 2021 Conference on Empirical Methods in Natural Language Processing,2021:2418-2431.

[1]	李清风, 金柳, 马慧芳, 张若一. 双视图对比学习引导的多行为推荐方法[J]. 计算机工程与科学, 2024, 46(04): 707-715.
[2]	崔大为，窦文华，计永兴. 三维视频压缩研究进展[J]. J4, 2011, 33(5): 63-68.
[3]	马颖哲，孙劲光. 一种多视角人脸检测方法[J]. J4, 2011, 33(1): 132-137.