基于Wasserstein GAN的文档表示模型

计算机工程与科学

基于Wasserstein GAN的文档表示模型

马永军，李亚军，汪睿，陈海山

（天津科技大学计算机科学与信息工程学院，天津 300457）

收稿日期:2018-01-22 修回日期:2018-02-28 出版日期:2019-01-25 发布日期:2019-01-25
基金资助:
天津市科技计划项目（17KPXMSF00140）；天津市教委社科重大项目（2017JWZD19）

A document representation model based on Wasserstein GAN

MA Yongjun,LI Yajun,WANG Rui,CHEN Haishan

（College of Computer Science and Information Engineering，Tianjin University of Science & Technology，Tianjin 300457，China)

Received:2018-01-22 Revised:2018-02-28 Online:2019-01-25 Published:2019-01-25

摘要/Abstract

摘要：

文档表示模型可以将非结构化的文本数据转化为结构化数据，是多种自然语言处理任务的基础，而目前基于词的模型在文档表示任务中有着无法直接表示文档的缺陷。针对此问题，基于生成对抗网络GAN可以使用两个神经网络进行对抗学习，从而很好地学习到原始数据分布的特点，
提出了文档表示模型WADM，使用去噪自编码器作为其判别网络，由其隐层直接得到文档的分布表示。实验表明，WADM能够准确抽取文档特征，相比基于词的模型具有更强的文档表示能力。

关键词: 文档表示, 生成对抗网络, 去噪自编码器, 神经网络

Abstract:

Document representation models can convert unstructured text data into structured data, which is the basis of many natural language processing tasks. Currently, wordbased models cannot deal with unregistered words and documents in the document representation tasks. The generative adversarial network (GAN) can use two neural networks to deal with confrontation so as to learn the distribution of the original data well. We propose a Wasserstein adversarial document model (WADM), which uses denoising autoencoder as its discriminant network and obtains document representation directly by its hidden layer. Experiments show that the WADM can extract document features accurately and has stronger document representation capability than word-based models.

Key words: document representation, generative adversarial network(GAN), denoising autoencoder, neural network

马永军，李亚军，汪睿，陈海山. 基于Wasserstein GAN的文档表示模型[J]. 计算机工程与科学.

MA Yongjun,LI Yajun,WANG Rui,CHEN Haishan. A document representation model based on Wasserstein GAN[J]. Computer Engineering & Science.

参考文献［3］

	［1］
	Xue Tao, Wang Yaling, Mu Nan. Convolutional neural network based on word sense disambiguation for text classification［J］.Application Research of Computers, 2018,35(10):28982903.(in Chinese)
［2］	Socher R,Perelygin A,Wu J Y,et al.Recursive deep models for semantic compositionality over a sentiment treebank［C］∥Proc of the 2013 Conference on Empirical Methods in Natural Language,2013:16311642.
［3］	Jiang Zhenchao, Li Lishuang, Huang Degen. Word representation based on word relations［J］.Journal of Chinese Information Processing,2017,31(3):2531.(in Chinese)
［4］	Bengio Y,Courville A C,Vincent P.Representation learning:A review and new perspectives［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):17981828.
［5］	BaezaYates R A,RibeiroNeto B.Modern information retrieval［M］.New York:ACM Press,1999.
［6］	Lee D L,Chuang H,Seamons K.Document ranking and the vectorspace model［J］.IEEE Software,1997,14(2):6775.
［7］	Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality［C］∥
	Proc of the 26th International Conference on Neural Information Processing Systems,2013:31113119.
［8］	Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space［J］.arXiv:1301.3781,2013.
［9］	Blei D M,Ng A Y,Jordan M I.Latent dirichlet allocation［J］.Journal of Machine Learning Research,2003,3:9931022.
［10］	Salakhutdinov R,Hinton G.Replicated Softmax:An undirected topic model［C］∥Proc of the 22nd International Conference on Neural Information Processing Systems,2009:16071614.
［11］	Hinton G E.Training products of experts by minimizing contrastive divergence［J］.Neural Computation,2002,14(8):17711800.
［12］	Larochelle H,Lauly S.A neural autoregressive topic model［C］∥Proc of the 25th International Conference on Neural Information Processing Systems,2012:27082716.
［13］	Larochelle H,Murray I.The neural autoregressive distribution estimator［J］.Journal of Machine Learning Research,2011,15:2937.
［14］	Le Q V,Mikolov T.Distributed representations of sentences and documents［C］∥Proc of the 31st International Conference on Machine Learning,2014:11881196.
［15］	Arjovsky M,Chintala S,Bottou L.Wasserstein GAN［J］. arXiv:1701.07875,2017.
［16］	Vincent P,Larochelle H,Bengio Y,et al.Extracting and composing robust features with denoising autoencoders［C］∥Proc of the 25th International Conference on Machine Learning,2008:10961103.
［17］	Goodfellow I J,PougetAbadie J,Mirza M,et al.Generative adversarial networks［C］∥Proc of the 27th International Conference on
	Neural Information Processing Systems,2014:26722680.
［18］	Tieleman T,Hinton G.Lecture 6.5rmsprop:Divide the gradient by a running average of its recent magnitude［J］.COURSERA:Neural Networks for Machine Learning,2012,4(2):2631.
［19］	20 Newsgroups ［EB/OL］.［20080114］.http://qwone.com/~jason/20Newsgroups/.
［20］	Ioffe S, Szegedy C. Batch normalization:Accelerating deep
	network training by reducing internal covariate shift［C］∥Proc of the 32nd International Conference on Machine Learning,2015:448456.
［21］	Xing C,Wang D,Zhang X W,et al.Document classification with distributions of word vectors［C］∥Proc of AsiaPacific Signal and Information Processing Association Annual Summit and Conference,2014:15.
	附中文参考文献：
［1］	薛涛,王雅玲,穆楠.基于词义消歧的卷积神经网络文本分类模型［J］.计算机应用研究,2018,35(10):28982903.
［3］	蒋振超,李丽双,黄德根.基于词语关系的词向量模型［J］.中文信息学报,2017,31(3):2531.

[1]	黄瑶, 潘丽丽, 熊思宇, 蒋湘辉, 马俊勇. 基于生成对抗网络与多头注意力的文本隐写术[J]. 计算机工程与科学, 2023, 45(10): 1789-1796.
[2]	周菊香, 周明涛, 甘健侯, 徐坚. 多阶段时序和语义信息增强的问题生成模型[J]. 计算机工程与科学, 2023, 45(10): 1847-1857.
[3]	杨春霞, 桂强, 马文文, 徐奔, . 融合图游走信息的图注意力网络方面级情感分析[J]. 计算机工程与科学, 2023, 45(10): 1858-1865.
[4]	刘通, 周宁宁. 基于Setwise排序的深度输入感知因子分解机[J]. 计算机工程与科学, 2023, 45(10): 1891-1900.
[5]	任拓, 闫玮, 况立群, 谢剑斌, 谌钟毓, 高峰, 郭锐, 束伟, 谢昌颐. 基于多对抗性鉴别网络的人脸活体检测[J]. 计算机工程与科学, 2023, 45(09): 1611-1620.
[6]	余子丞, 凌捷. 基于Transformer和多特征融合的DGA域名检测方法[J]. 计算机工程与科学, 2023, 45(08): 1416-1423.
[7]	刘俊奇, 涂文轩, 祝恩. 图卷积神经网络综述[J]. 计算机工程与科学, 2023, 45(08): 1472-1481.
[8]	易啸, 马胜, 肖侬. 深度学习加速器在不同剪枝策略下的运行优化[J]. 计算机工程与科学, 2023, 45(07): 1141-1148.
[9]	刘阳, 粟航, 何倩, 申普, 刘鹏. 基于云-边协同变分自编码神经网络的设备故障检测方法[J]. 计算机工程与科学, 2023, 45(07): 1188-1196.
[10]	崔克彬, 崔叶微. 基于卷积和Transformer的断路器动触头跟踪方法研究[J]. 计算机工程与科学, 2023, 45(07): 1236-1244.
[11]	王剑, 姜林, 王琳钦, 余正涛, 张松, 高盛祥, . 基于BiLSTM的低资源老挝语文本正则化任务[J]. 计算机工程与科学, 2023, 45(07): 1292-1299.
[12]	曹健, 陈怡梅, 李海生, 蔡强, . 基于图神经网络的行人轨迹预测研究综述[J]. 计算机工程与科学, 2023, 45(06): 1040-1053.
[13]	余泽鹏, 安业腾, 张烁, 杨自兴, 陆继翔, 曹蓉蓉, 陈轶洲, 李文中, 陆桑璐. 基于会话嵌入的应用程序使用预测[J]. 计算机工程与科学, 2023, 45(06): 1079-1086.
[14]	王扬, 陈智斌. 一种求解CVRP的动态图转换模型[J]. 计算机工程与科学, 2023, 45(05): 859-868.
[15]	排日旦·阿布都热依木, 吐尔地·托合提, 艾斯卡尔·艾木都拉, . 基于深度学习的实体关系抽取方法研究[J]. 计算机工程与科学, 2023, 45(05): 895-902.