关键词: 神经网络, 神经机器翻译, 信息传递, 信息退化, 残差网络, 门机制

Abstract: In the field of Neural Machine Translation (NMT), the multi-layer neural network model structure can significantly improve the translation performance. However, the structure of multi-layer neural network has an inherent problem with information transfer degeneracy. To alleviate this problem, this paper proposes an information transfer enhancement method by fusing layers information and sublayers information. By introducing a "retention gate" mechanism to control the fused information transfer weight, which is aggregated with the output of the current layer and then serves as the input of the next layer, thus making fuller information transfer between layers. Experiments were carried out on the most advanced NMT model Transformer. Experimental results on the Chinese-English and German-English tasks show that our method improves BLEU score by 0.66, and 0.42 in comparison to the baseline system. 

Key words: neural network, neural machine translation, information transfer, information degene- , racy, residual network, gate mechanism