Abstract: This paper proposes a reinforcement learning-based method for generating adversarial examples against PE malware. Firstly, it regards the generation of adversarial examples for PE malware as a sequence-to-sequence generation task, which models sequences on an offline reinforcement learning dataset and leverages the powerful sequence generation capability of Transformer by incrementally generating sequences through predicting actions at each step. Furthermore, an information transmission mechanism is introduced to facilitate cross-episode information transfer during the reinforcement learning process, enhancing data efficiency. Experimental results demonstrate that the evasion rate of PE malware adversarial examples generated using this method outperforms those in comparative experiments and exhibits transferability.

Key words: reinforcement learning, adversarial example, PE malware, malware detection