A survey of Chinese text classification based on deep learning

Abstract

Abstract: In the era of big data, with the continuous popularization of social media, various text data are growing in the network and in life. It is of great significance to analyze and manage text data using text classification technology. Text classification is a basic research field in the field of artificial intelligence natural language processing. Under the given criteria, it classifies text according to content. The application scenarios of text classification are very extensive, such as sentiment analysis, topic classification, relationship classification, etc. Deep learning is a method of representation learning based on data in machine learning, and it shows good classification effect in text data processing. Chinese text and English text have differences in form, sound, and image. Focusing on the uniqueness of Chinese text classification, this paper analyzes and expounds the deep learning methods used for Chinese text classification, and finally sorts out commonly used datasets for Chinese text classification.

Key words: Chinese text classification, natural language, deep learning, machine learning

GAO Shan, LI Shi-jie, CAI Zhi-ping. A survey of Chinese text classification based on deep learning[J]. Computer Engineering & Science, 2024, 46(04): 684-692.

References ［66］

［1］	Zhang W X, Li X, Deng Y, et al. A survey on aspect-based sentiment analysis: Tasks, methods, and challenges［J］. arXiv:2203.01054,2022.
［2］	Kalchbrenner N,Grefenstette E,Blunsom P.A convolutional neural network for modelling sentences［C］∥Proc of the 52nd Annual Meeting of the Association for Computational Linguistics,2014:655-665.
［3］	Liu P,Qiu X,Chen X,et al.Multi-timescale long short-term memory neural network for modelling sentences and documents［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2015:2326-2335.
［4］	Lee J Y,Dernoncourt F.Sequential short-text classification with recurrent and convolutional neural networks［C］∥Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2016:515-520.
［5］	Chen J,Gong Z,Liu W.A nonparametric model for online topic discovery with word embeddings［J］.Information Sciences, 2019，504:32-47.
［6］	Eddy S R.Hidden Markov models［J］.Current Opinion in Structural Biology,1996,6(3):361-365.
［7］	Lafferty J,Mccallum A,Pereira F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data［C］∥Proc of the 18th International Conference on Machine Learning,2001:282-289．
［8］	Yang Y, Ren G.HanLP-based technology function matrix construction on Chinese process patents［J］.International Journal of Mobile Computing and Multimedia Communications,2020,11(3):48-64.
［9］	Peng N，Dredze M.Multi-task domain adaptation for sequence tagging［C］∥Proc of the 2nd Workshop on Representation Learning for Natural Language Processing,2017:91-100.
［10］	Zhang Y,Jin R,Zhou Z H.Understanding bag-of-words model:A statistical framework［J］.International Journal of Machine Learning & Cybernetics,2010,1:43-52.
［11］	Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space［J］.arXiv:1301.3781,2013.
［12］	Pennington J,Socher R,Manning C.GloVe:Global vectors for word representation［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2014:1532-1543.
［13］	Joulin A,Grave E,Bojanowski P,et al.Bag of tricks for efficient text classification［J］.arXiv:1607.01759,2016.
［14］	Maron M E.Automatic indexing:An experimental inquiry［J］.ACM,1961,8(3):404-417.
［15］	Cover T M,Hart P E.Nearest neighbor pattern classification［J］.IEEE Transactions on Information Theory,1967,13(1):21-27.
［16］	Joachims T. Text categorization with support vector machines:Learning with many relevant features［C］∥Proc of European Conference on Machine Learning,1998:137-142.
［17］	Aly R,Remus S, Biemann C.Hierarchical multi-label classification of text with capsule networks［C］∥Proc of the 57th Conference of the Association for Computational Linguistics,2019:323-330.
［18］	刘鑫昊,谭庆平,曾平,等.几种基于MOOC的文本分类算法的比较与实现［J］.软件,2016,37(9):27-33.
	Liu Xin-hao,Tan Qing-ping,Zeng Ping,et al.Comparison and implementation of several text classification algorithms based on MOOC ［J］.Software,2016,37(9):27-33.
［19］	Chen T,Guestrin C.XGBoost:A scalable tree boosting system［C］∥Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2016:785-794.
［20］	Ke G,Meng Q,Finley T,et al.LightGBM:A highly efficient gradient boosting decision tree［C］∥Proc of the 31st International Conference on Neural Information Processing Systems,2017:3149-3157.
［21］	Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks［J］.Science,2006，313(5786):504-507．
［22］	Soni S,Chouhan S S,Rathore S S.TextConvoNet:A convolutional neural network based architecture for text classification［J］.arXiv:2203.05173,2022.
［23］	王芝辉,王晓东.基于神经网络的文本分类方法研究［J］.计算机工程,2020,46(3):11-17.
	Wang Zhi-hui,Wang Xiao-dong.Research on text classification methods based on neural networks ［J］. Computer Engineering,2020,46(3):11-17.
［24］	Hochreiter S,Schmidhuber J.Long short-term memory［J］.Neural Computation,1997,9(8):1735-1780.
［25］	Kim Y.Convolutional neural networks for sentence classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2014:1746-1751.
［26］	Alexis C, Holger S, Loc B, et al. Very deep convolutional networks for text classification［C］∥Proc of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017:1107-1116.
［27］	Johnson R, Zhang T. Semi-supervised convolutional neural networks for text categorization via region embedding［C］∥Proc of the 28th International Conference on Neural Information Processing Systems,2015:919-927.
［28］	Johnson R,Zhang T. Deep pyramid convolutional neural networks for text categorization［C］∥Proc of the 55th Annual Meeting of the Association for Computational Linguistics,2017:562-570.
［29］	He K,Zhang X,Ren S,et al.Identity mappings in deep residual networks［C］∥Proc of the 14th European Conference on Computer Vision,2016:630-645.
［30］	Zhang X, Zhao J,Lecun Y.Character-level convolutional networks for text classification［J］.arXiv:1509.01626,2015.
［31］	Adams B,McKenzie G.Crowdsourcing the character of a place:Character-level convolutional networks for multilingual geographic text classification［J］.Transactions in GIS,2018,22(2):394-408.
［32］	Kipf T N, Welling M.Semi-supervised classification with graph convolutional networks［J］.arXiv:1609.02907,2016.
［33］	Yao L,Mao C S,Luo Y.Graph convolutional networks for text classification［C］∥Proc of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence,2019:7370-7377．
［34］	Chen J,Ma T,Xiao C.FastGCN:Fast learning with graph convolutional networks via importance sampling［J］.arXiv：1801.10247,2018.
［35］	Liu X,You X,Zhang X，et al.Tensor graph convolutional networks for text classification［J］.arXiv:2001.05313v1,2020.
［36］	Huang L,Ma D,Li S,et al.Text level graph neural network for text classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:3444-3450.
［37］	Chen G,Tian Y,Song Y.Joint aspect extraction and sentiment analysis with directional graph convolutional networks［C］∥Proc of the International Conference on Computational Linguistics,2020:272-279.
［38］	Chen M,Wei Z,Huang Z,et al.Simple and deep graph con- volutional networks［J］.arXiv:2007.02133,2020.
［39］	Chung J,Gulcehre C,Cho K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling［J］.arXiv:1412.3555,2014.
［40］	Koutnik J,Greff K,Gomez F,et al.A clockwork RNN［J］.arXiv:1402.3511,2014.
［41］	Dieng A B,Wang C,Gao J F,et al.TopicRNN:A recurrent neural network with long-range semantic dependency［J］.arXiv:1611.01702,2017.
［42］	Schuster M，Paliwal K K.Bidirectional recurrent neural networks［J］.IEEE Transactions on Signal Processing,1977,45(11):2673-2681．
［43］	Wang Y Q,Sun A X,Han J L,et al.Sentiment analysis by capsules［C］∥Proc of the 2018 World Wide Web Confe- rence,2018:1165-1174.
［44］	Tai K S, Socher R, Manning C D.Improved semantic representations from tree-structured long short-term memory networks［J］.Computer Science, 2015, 5(1):36.
［45］	袁婷婷,杨文忠,仲丽君,等.基于性格的微博情感分析模型PLSTM［J］.计算机应用研究，2020,37(2):342-346.
	Yuan Ting-ting, Yang Wen-zhong, Zhong Li-jun, et al. Personality-based microblog sentiment analysis model PLSTM［J］.Application Research of Computers,2020,37(2):342-346.
［46］	孙明敏.基于GRU-Attention的中文文本分类［J］.现代信息科技,2019,3(3):10-12.
	Sun Ming-min.Chinese text classification based on GRU- Attention ［J］. Modern Information Technology,2019,3(3):10-12.
［47］	Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate［J］.arXiv:1409.0473,2014.
［48］	Yang Z,Yang D,Dyer C,et al.Hierarchical attention networks for document classification［C］∥Proc of the Confe- rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technolo- gies,2016:1480-1489.
［49］	Qiu X,Sun T,Xu Y,et al.Pre-trained models for natural language processing:A survey［J］.Science China Technolo- gical Sciences,2020,63(10):1871-1897.
［50］	Devlin J,Chang M,Lee K,et al.BERT:Pre-training of deep bidirectional transformers for language understanding［C］∥Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019:4171-4186.
［51］	Liu Y,Ott M,Goyal N,et al.RoBERTa:A robustly optimized BERT pretraining approach［J］.arXiv:1907.11692,2019.
［52］	Lan Z,Chen M,Goodman S,et al.ALBERT:A lite BERT for self-supervised learning of language representations［J］.arXiv:1909.11942,2019.
［53］	Shin H C,Ihsani A,Mandava S,et al.GANBERT:Generative adversarial networks with bidirectional encoder representations from transformers for MRI to PET synthesis［J］.arXiv:2008.04393,2020.
［54］	Su H,Shi W,Shen X,et al.RoCBert:Robust Chinese Bert with multimodal contrastive pretraining［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:921-931.
［55］	Dai Y,Li L Y,Zhou C,et al.“Is whole word masking always better for Chinese BERT?”:Probing on Chinese grammatical error correction［C］∥Proc of Findings of the Association for Computational Linguistics,2022:1-8.
［56］	Yu W H,Zhu C G,Fang Y W,et al.Dict-BERT:Enhancing language model pre-training with dictionary［C］∥Proc of Findings of the Association for Computational Linguistics,2022:1907-1918.
［57］	Shehab A,Ahmed R.Performance study on extractive text summarization using BERT models［J］.Information,2022,13(2):67-71.
［58］	Jiao X,Yin Y,Shang L,et al.TinyBERT:Distilling BERT for natural language understanding［C］∥Proc of Findings of the Association for Computational Linguistics,2020:4163-4174.
［59］	Deguang C,Jie Z.LightMobileBert:A secondary lightweight model based on MobileBert［J］.Journal of Intelligent & Fuzzy Systems,2023,44(2):2117-2129.
［60］	Chen C,Yin Y,Shang L,et al.bert2BERT:Towards reusable pretrained language models［C］∥Proc of the 60th Annual Meeting of the Association for Computational Linguistics,2022:2134-2148.
［61］	Peters M E,Neumann M,Iyyer M,et al.Deep contextualized word representations［C］∥Proc of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2018:2227-2237.
［62］	Howard J, Ruder S. Universal language model fine-tuning for text classification［C］∥Proc of the 56th Annual Meeting of the Association for Computational Linguistics,2018:328-339.
［63］	Yang Z L,Dai Z H,Yang Y M,et al.XLNet:Generalized autoregressive pretraining for language understanding［C］∥Proc of the 33rd International Conference on Neural Information Processing Systems,2019:5753-5763.
［64］	Zhang H,Zhang J.Text graph transformer for document classification［C］∥Proc of the Conference on Empirical Methods in Natural Language Processing,2020:8322-8327.
［65］	Chang W-C,Yu H-F,Zhong K, et al. Taming pretrained transformers for extreme multi-label text classification［C］∥Proc of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,2020:3163-3171.
［66］	Jiang T,Wang D,Sun L,et al.LightXML:Transformer with dynamic negative sampling for high-performance extreme multi-label text classification［J］.arXiv:2101.03305,2021.

[1]	YU Tian-ci, GAO Shang. A code summarization generation model fusing multi-structure data [J]. Computer Engineering & Science, 2024, 46(04): 667-675.
[2]	LUO Yue-tong, LI Chao, ZHOU Bo, ZHANG Yan-kong. An interactive separation method for confusable defects in industrial defect classification [J]. Computer Engineering & Science, 2024, 46(03): 463-470.
[3]	Lv Fu, HAN Xiao-tian, FENG Yong-an, XIANG Liang. A texture image classification method based on adaptive texture feature fusion [J]. Computer Engineering & Science, 2024, 46(03): 488-498.
[4]	JI Xu-rui, WEI De-jian, ZHANG Jun-zhong, ZHANG Shuai, CAO Hui. Research progress on information extraction methods of Chinese electronic medical records [J]. Computer Engineering & Science, 2024, 46(02): 325-337.
[5]	HUANG Ze-biao, DONG De-zun, QI Xing-yun. Gloo+: Accelerating distributed training of deep learning using in-network computing [J]. Computer Engineering & Science, 2024, 46(01): 28-36.
[6]	QIU Xiao-meng, WANG Lin, GU Wen-jun, SONG Wei, TIAN Hao-lai, HU Yu. A time series image semantic segmentation model modified by optical flow [J]. Computer Engineering & Science, 2024, 46(01): 102-110.
[7]	CUI Hao, WAN Ya-ping, ZHONG Hua, NIE Ming-xing, XIAO Yang. Human activity recognition based on LoRa devices [J]. Computer Engineering & Science, 2024, 46(01): 111-121.
[8]	PENG Chang, LIU Qing-zhi, CHEN Chang-bo, . Loop permutation and auto-tuning under polyhedral model [J]. Computer Engineering & Science, 2023, 45(12): 2121-2134.
[9]	ZHANG Qian, CHEN Zi-qiang, SUN Zong-wei, LAI Jing-an. A fog target detection algorithm fusing high-resolution network [J]. Computer Engineering & Science, 2023, 45(11): 1970-1981.
[10]	LIU Yu-mo, LIU Jian-fei, HAO Lu-guo, ZENG Wen-bin. A multi-scale feature fusion network based fast CU partitioning in HEVC intra coding [J]. Computer Engineering & Science, 2023, 45(11): 1991-1998.
[11]	LI Zhuo-xuan, ZHOU Ya-tong. iSFF-DBNet:An improved text detection algorithm in e-commerce images [J]. Computer Engineering & Science, 2023, 45(11): 2008-2017.
[12]	MA Zhi-feng, ZHANG Hao, LIU Jie. A survey of precipitation nowcasting based on deep learning [J]. Computer Engineering & Science, 2023, 45(10): 1731-1753.
[13]	GUO Yi, HE Ting-nian, LI Ai-bin, MAO Jun-yu. A knowledge tracing model fusing GA-CART and Deep-IRT [J]. Computer Engineering & Science, 2023, 45(09): 1691-1700.
[14]	ZHAO Zhen-yu, YANG Tian-hao, JIANG Wen-cheng, ZHANG Shu-zheng. A machine learning-based fast calculation method of multi-voltage, multi-temperature and multi-parameter standard cell delay [J]. Computer Engineering & Science, 2023, 45(08): 1331-1338.
[15]	MA Si-yuan, JIAO Jia-hui, REN Sheng-qi, SONG Wei. Missing value filling for multi-variable urban air quality data based on attention mechanism [J]. Computer Engineering & Science, 2023, 45(08): 1354-1364.

A survey of Chinese text classification based on deep learning

PDF

Knowledge

Abstract

Cite this article

share this article

References ［66］

Related Articles 15

Recommended Articles

Metrics

Comments