关键词: 文本隐写, 暴露偏差, 生成对抗网络, 多头注意力

Abstract: With the development of deep learning, steganography based on text generation has made significant break-throughs. Existing text-based steganography methods suffer from exposure bias, where the input during training comes from real sample labels, while the input during prediction comes from the output predicted in the previous time step. This difference in input samples between training and prediction leads to error accumulation, resulting in a large distribution difference between generated samples and real samples. To address this problem, this paper proposes a text steganography model called TS-GANMA based on generative adversarial networks and multi-head attention. First, a text generator is trained using a generative adversarial network, and multi-head attention mechanisms are used to extract multi-head attention scores to participate in the reward calculation of the reward module, obtaining feedback information more suitable for the generator. Then, the generator and discriminator are trained in an adversarial manner, which can solve the exposure bias problem and optimize the text generation model. Finally, the conditional probability distribution output by the text generation model is encoded to embed secret information.  The experimental results  demonstrate that the steganography method based on TS-GANMA has a much lower perplexity than the methods based on LSTM-vlc and ADG at the same embedding rate. This is because the steganographic text generated by the TS-GANMA model fits the statistical distribution of the real text better, and can generate higher quality steganographic text.

Key words: text steganography, exposure bias, generative adversarial network, multi-head attention