基于深度学习的中文文本分类综述

摘要/Abstract

摘要： 大数据时代，随着社交媒体的不断普及，在网络以及生活中，各类文本数据日益增长，采用文本分类技术对文本数据进行分析和管理具有重要的意义。文本分类是自然语言处理领域中的一个基础研究内容，在给定标准下，根据内容对文本进行分类，文本分类的场景应用十分广泛，如情感分析、话题分类和关系分类等。深度学习是机器学习中一种基于对数据进行表征学习的方法，在文本数据处理中表现出了较好的分类效果。中文文本与英文文本在形、音、象上都有着区别，着眼于中文文本分类的特别之处，对用于中文文本分类的深度学习方法进行分析与阐述，最终梳理出常用于中文文本分类的数据集。

关键词: 中文文本分类, 自然语言, 深度学习, 机器学习

Abstract: In the era of big data, with the continuous popularization of social media, various text data are growing in the network and in life. It is of great significance to analyze and manage text data using text classification technology. Text classification is a basic research field in the field of artificial intelligence natural language processing. Under the given criteria, it classifies text according to content. The application scenarios of text classification are very extensive, such as sentiment analysis, topic classification, relationship classification, etc. Deep learning is a method of representation learning based on data in machine learning, and it shows good classification effect in text data processing. Chinese text and English text have differences in form, sound, and image. Focusing on the uniqueness of Chinese text classification, this paper analyzes and expounds the deep learning methods used for Chinese text classification, and finally sorts out commonly used datasets for Chinese text classification.

Key words: Chinese text classification, natural language, deep learning, machine learning

高珊, 李世杰, 蔡志平. 基于深度学习的中文文本分类综述[J]. 计算机工程与科学, 2024, 46(04): 684-692.

GAO Shan, LI Shi-jie, CAI Zhi-ping. A survey of Chinese text classification based on deep learning[J]. Computer Engineering & Science, 2024, 46(04): 684-692.

参考文献［66］

［1］	Zhang W X, Li X, Deng Y, et al. A survey on aspect-based sentiment analysis: Tasks, methods, and challenges［J］. arXiv:2203.01054,2022.
［2］	Kalchbrenner N,Grefenstette E,Blunsom P.A convolutional neural network for modelling sentences［C］∥Proc of the 52nd Annual Meeting of the Association for Computational Linguistics,2014:655-665.
［3］	Liu P,Qiu X,Chen X,et al.Multi-timescale long short-term memory neural network for modelling sentences and documents［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2015:2326-2335.
［4］	Lee J Y,Dernoncourt F.Sequential short-text classification with recurrent and convolutional neural networks［C］∥Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2016:515-520.
［5］	Chen J,Gong Z,Liu W.A nonparametric model for online topic discovery with word embeddings［J］.Information Sciences, 2019，504:32-47.
［6］	Eddy S R.Hidden Markov models［J］.Current Opinion in Structural Biology,1996,6(3):361-365.
［7］	Lafferty J,Mccallum A,Pereira F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data［C］∥Proc of the 18th International Conference on Machine Learning,2001:282-289．
［8］	Yang Y, Ren G.HanLP-based technology function matrix construction on Chinese process patents［J］.International Journal of Mobile Computing and Multimedia Communications,2020,11(3):48-64.
［9］	Peng N，Dredze M.Multi-task domain adaptation for sequence tagging［C］∥Proc of the 2nd Workshop on Representation Learning for Natural Language Processing,2017:91-100.
［10］	Zhang Y,Jin R,Zhou Z H.Understanding bag-of-words model:A statistical framework［J］.International Journal of Machine Learning & Cybernetics,2010,1:43-52.
［11］	Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space［J］.arXiv:1301.3781,2013.
［12］	Pennington J,Socher R,Manning C.GloVe:Global vectors for word representation［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2014:1532-1543.
［13］	Joulin A,Grave E,Bojanowski P,et al.Bag of tricks for efficient text classification［J］.arXiv:1607.01759,2016.
［14］	Maron M E.Automatic indexing:An experimental inquiry［J］.ACM,1961,8(3):404-417.
［15］	Cover T M,Hart P E.Nearest neighbor pattern classification［J］.IEEE Transactions on Information Theory,1967,13(1):21-27.
［16］	Joachims T. Text categorization with support vector machines:Learning with many relevant features［C］∥Proc of European Conference on Machine Learning,1998:137-142.
［17］	Aly R,Remus S, Biemann C.Hierarchical multi-label classification of text with capsule networks［C］∥Proc of the 57th Conference of the Association for Computational Linguistics,2019:323-330.
［18］	刘鑫昊,谭庆平,曾平,等.几种基于MOOC的文本分类算法的比较与实现［J］.软件,2016,37(9):27-33.
	Liu Xin-hao,Tan Qing-ping,Zeng Ping,et al.Comparison and implementation of several text classification algorithms based on MOOC ［J］.Software,2016,37(9):27-33.
［19］	Chen T,Guestrin C.XGBoost:A scalable tree boosting system［C］∥Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2016:785-794.
［20］	Ke G,Meng Q,Finley T,et al.LightGBM:A highly efficient gradient boosting decision tree［C］∥Proc of the 31st International Conference on Neural Information Processing Systems,2017:3149-3157.
［21］	Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks［J］.Science,2006，313(5786):504-507．
［22］	Soni S,Chouhan S S,Rathore S S.TextConvoNet:A convolutional neural network based architecture for text classification［J］.arXiv:2203.05173,2022.
［23］	王芝辉,王晓东.基于神经网络的文本分类方法研究［J］.计算机工程,2020,46(3):11-17.
	Wang Zhi-hui,Wang Xiao-dong.Research on text classification methods based on neural networks ［J］. Computer Engineering,2020,46(3):11-17.
［24］	Hochreiter S,Schmidhuber J.Long short-term memory［J］.Neural Computation,1997,9(8):1735-1780.
［25］	Kim Y.Convolutional neural networks for sentence classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2014:1746-1751.
［26］	Alexis C, Holger S, Loc B, et al. Very deep convolutional networks for text classification［C］∥Proc of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017:1107-1116.
［27］	Johnson R, Zhang T. Semi-supervised convolutional neural networks for text categorization via region embedding［C］∥Proc of the 28th International Conference on Neural Information Processing Systems,2015:919-927.
［28］	Johnson R,Zhang T. Deep pyramid convolutional neural networks for text categorization［C］∥Proc of the 55th Annual Meeting of the Association for Computational Linguistics,2017:562-570.
［29］	He K,Zhang X,Ren S,et al.Identity mappings in deep residual networks［C］∥Proc of the 14th European Conference on Computer Vision,2016:630-645.
［30］	Zhang X, Zhao J,Lecun Y.Character-level convolutional networks for text classification［J］.arXiv:1509.01626,2015.
［31］	Adams B,McKenzie G.Crowdsourcing the character of a place:Character-level convolutional networks for multilingual geographic text classification［J］.Transactions in GIS,2018,22(2):394-408.
［32］	Kipf T N, Welling M.Semi-supervised classification with graph convolutional networks［J］.arXiv:1609.02907,2016.
［33］	Yao L,Mao C S,Luo Y.Graph convolutional networks for text classification［C］∥Proc of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence,2019:7370-7377．
［34］	Chen J,Ma T,Xiao C.FastGCN:Fast learning with graph convolutional networks via importance sampling［J］.arXiv：1801.10247,2018.
［35］	Liu X,You X,Zhang X，et al.Tensor graph convolutional networks for text classification［J］.arXiv:2001.05313v1,2020.
［36］	Huang L,Ma D,Li S,et al.Text level graph neural network for text classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:3444-3450.
［37］	Chen G,Tian Y,Song Y.Joint aspect extraction and sentiment analysis with directional graph convolutional networks［C］∥Proc of the International Conference on Computational Linguistics,2020:272-279.
［38］	Chen M,Wei Z,Huang Z,et al.Simple and deep graph con- volutional networks［J］.arXiv:2007.02133,2020.
［39］	Chung J,Gulcehre C,Cho K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling［J］.arXiv:1412.3555,2014.
［40］	Koutnik J,Greff K,Gomez F,et al.A clockwork RNN［J］.arXiv:1402.3511,2014.
［41］	Dieng A B,Wang C,Gao J F,et al.TopicRNN:A recurrent neural network with long-range semantic dependency［J］.arXiv:1611.01702,2017.
［42］	Schuster M，Paliwal K K.Bidirectional recurrent neural networks［J］.IEEE Transactions on Signal Processing,1977,45(11):2673-2681．
［43］	Wang Y Q,Sun A X,Han J L,et al.Sentiment analysis by capsules［C］∥Proc of the 2018 World Wide Web Confe- rence,2018:1165-1174.
［44］	Tai K S, Socher R, Manning C D.Improved semantic representations from tree-structured long short-term memory networks［J］.Computer Science, 2015, 5(1):36.
［45］	袁婷婷,杨文忠,仲丽君,等.基于性格的微博情感分析模型PLSTM［J］.计算机应用研究，2020,37(2):342-346.
	Yuan Ting-ting, Yang Wen-zhong, Zhong Li-jun, et al. Personality-based microblog sentiment analysis model PLSTM［J］.Application Research of Computers,2020,37(2):342-346.
［46］	孙明敏.基于GRU-Attention的中文文本分类［J］.现代信息科技,2019,3(3):10-12.
	Sun Ming-min.Chinese text classification based on GRU- Attention ［J］. Modern Information Technology,2019,3(3):10-12.
［47］	Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate［J］.arXiv:1409.0473,2014.
［48］	Yang Z,Yang D,Dyer C,et al.Hierarchical attention networks for document classification［C］∥Proc of the Confe- rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technolo- gies,2016:1480-1489.
［49］	Qiu X,Sun T,Xu Y,et al.Pre-trained models for natural language processing:A survey［J］.Science China Technolo- gical Sciences,2020,63(10):1871-1897.
［50］	Devlin J,Chang M,Lee K,et al.BERT:Pre-training of deep bidirectional transformers for language understanding［C］∥Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019:4171-4186.
［51］	Liu Y,Ott M,Goyal N,et al.RoBERTa:A robustly optimized BERT pretraining approach［J］.arXiv:1907.11692,2019.
［52］	Lan Z,Chen M,Goodman S,et al.ALBERT:A lite BERT for self-supervised learning of language representations［J］.arXiv:1909.11942,2019.
［53］	Shin H C,Ihsani A,Mandava S,et al.GANBERT:Generative adversarial networks with bidirectional encoder representations from transformers for MRI to PET synthesis［J］.arXiv:2008.04393,2020.
［54］	Su H,Shi W,Shen X,et al.RoCBert:Robust Chinese Bert with multimodal contrastive pretraining［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:921-931.
［55］	Dai Y,Li L Y,Zhou C,et al.“Is whole word masking always better for Chinese BERT?”:Probing on Chinese grammatical error correction［C］∥Proc of Findings of the Association for Computational Linguistics,2022:1-8.
［56］	Yu W H,Zhu C G,Fang Y W,et al.Dict-BERT:Enhancing language model pre-training with dictionary［C］∥Proc of Findings of the Association for Computational Linguistics,2022:1907-1918.
［57］	Shehab A,Ahmed R.Performance study on extractive text summarization using BERT models［J］.Information,2022,13(2):67-71.
［58］	Jiao X,Yin Y,Shang L,et al.TinyBERT:Distilling BERT for natural language understanding［C］∥Proc of Findings of the Association for Computational Linguistics,2020:4163-4174.
［59］	Deguang C,Jie Z.LightMobileBert:A secondary lightweight model based on MobileBert［J］.Journal of Intelligent & Fuzzy Systems,2023,44(2):2117-2129.
［60］	Chen C,Yin Y,Shang L,et al.bert2BERT:Towards reusable pretrained language models［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:2134-2148.
［61］	Peters M E,Neumann M,Iyyer M,et al.Deep contextualized word representations［C］∥Proc of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2018:2227-2237.
［62］	Howard J, Ruder S. Universal language model fine-tuning for text classification［C］∥Proc of the 56th Annual Meeting of the Association for Computational Linguistics,2018:328-339.
［63］	Yang Z L,Dai Z H,Yang Y M,et al.XLNet:Generalized autoregressive pretraining for language understanding［C］∥Proc of the 33rd International Conference on Neural Information Processing Systems,2019:5753-5763.
［64］	Zhang H,Zhang J.Text graph transformer for document classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2020:8322-8327.
［65］	Chang W-C,Yu H-F,Zhong K, et al. Taming pretrained transformers for extreme multi-label text classification［C］∥Proc of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,2020:3163-3171.
［66］	Jiang T,Wang D,Sun L,et al.LightXML:Transformer with dynamic negative sampling for high-performance extreme multi-label text classification［J］.arXiv:2101.03305,2021.

[1]	余天赐, 高尚. 融合多结构信息的代码注释生成模型[J]. 计算机工程与科学, 2024, 46(04): 667-675.
[2]	罗月童, 李超, 周波, 张延孔. 面向工业缺陷分类的交互式易混淆缺陷分离方法研究[J]. 计算机工程与科学, 2024, 46(03): 463-470.
[3]	吕伏, 韩晓天, 冯永安, 项梁. 基于自适应纹理特征融合的纹理图像分类方法[J]. 计算机工程与科学, 2024, 46(03): 488-498.
[4]	吉旭瑞, 魏德健, 张俊忠, 张帅, 曹慧. 中文电子病历信息提取方法研究综述[J]. 计算机工程与科学, 2024, 46(02): 325-337.
[5]	黄泽彪, 董德尊, 齐星云. Gloo+：利用在网计算技术加速分布式深度学习训练[J]. 计算机工程与科学, 2024, 46(01): 28-36.
[6]	邱晓梦, 王琳, 谷文俊, 宋伟, 田浩来, 胡誉. 光流法修正的时序图像语义分割模型[J]. 计算机工程与科学, 2024, 46(01): 102-110.
[7]	崔浩, 万亚平, 钟华, 聂明星, 肖杨. 基于LoRa设备的人体活动识别研究[J]. 计算机工程与科学, 2024, 46(01): 111-121.
[8]	彭畅, 刘青枝, 陈长波, . 多面体模型下的循环置换与自动调优[J]. 计算机工程与科学, 2023, 45(12): 2121-2134.
[9]	张骞, 陈紫强, 孙宗威, 赖镜安. 融合高分辨率网络的雾天目标检测算法[J]. 计算机工程与科学, 2023, 45(11): 1970-1981.
[10]	刘雨墨, 刘剑飞, 郝禄国, 曾文彬. 基于多尺度特征融合网络的HEVC帧内编码单元快速划分研究[J]. 计算机工程与科学, 2023, 45(11): 1991-1998.
[11]	李卓璇, 周亚同. 改进DBNet的电商图像文字检测算法研究[J]. 计算机工程与科学, 2023, 45(11): 2008-2017.
[12]	马志峰, 张浩, 刘劼. 基于深度学习的短临降水预报综述[J]. 计算机工程与科学, 2023, 45(10): 1731-1753.
[13]	张耐, 张晨亮, 柳永翔, 陈聪, 黄艳婷. 基于模型混合的智能交易行为异常检测[J]. 计算机工程与科学, 2023, 45(09): 1639-1647.
[14]	郭艺, 何廷年, 李爱斌, 毛君宇. 融合GA-CART和Deep-IRT的知识追踪模型[J]. 计算机工程与科学, 2023, 45(09): 1691-1700.
[15]	赵振宇, 杨天豪, 蒋汶乘, 张书政. 基于机器学习的多压多温多参标准单元延迟快速计算方法[J]. 计算机工程与科学, 2023, 45(08): 1331-1338.