• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学

• 人工智能与数据挖掘 • 上一篇    下一篇

基于强化学习的体系对抗仿真战役层次指控算法

闫雪飞,李新明,刘东,刘德生,李强   

  1. (装备学院复杂电子系统仿真实验室,北京 101416)
  • 收稿日期:2017-04-24 修回日期:2017-06-28 出版日期:2018-08-25 发布日期:2018-08-25
  • 基金资助:

    装备预研领域基金项目(61400010103);重点实验室基础研究项目(DXZTJCZZ2015007)。

A RL-based command and control algorithm for
SoS confrontation simulation at the tactical level

YAN Xuefei,LI Xinming,LIU Dong,LIU Desheng,LI Qiang   

  1. (Laboratory of Science and Technology on Complex Electronic System Simulation,Equipment Academy,Beijing 101416,China)
  • Received:2017-04-24 Revised:2017-06-28 Online:2018-08-25 Published:2018-08-25

摘要:

针对传统的认知决策技术无法有效应对体系对抗环境具有的不确定性、未知性以及复杂性问题,提出一种基于强化学习(RL)的体系对抗仿真战役层次指控算法。介绍了包含侦察类、打击类、通信类、补给类、修复类以及指控类Agent的UML体系架构,对自主开发的作战仿真原型系统及其作战想定进行了说明,在对战役层次指控Agent认知域描述与假设的基础上,对改进Qlearning认知决策算法的参数归一化、基于GRBF神经网络的Q离散、基于TD公式的跨步差分机制以及网络结构的学习训练过程进行了详细说明。最后,通过地空一体化联合体系对抗仿真验证了算法的有效性,并通过对算法的大量可视化回溯分析发现,一定程度的火力协调以及不间断的战术机动对于作战效能的提升以及毁伤的减免具有重要的意义。

关键词: 武器装备体系, 作战仿真, 强化学习, GRBF神经网络, 认知决策

Abstract:

Aiming at the problem that the traditional cognition techniques are not adaptive to the uncertainty and complexity in the Weapon SystemofSystems (WSoS) confrontation environment, a Command and Control (C2) algorithm based on Reinforcement Learning (RL) is proposed for the WSoS confrontation simulation at the tactical level. The UML architecture of WSoS that consists of a communication class, scouting class, attacking class, command class, supplying class and repairing class is designed and the battle simulation platform with the battle scenario is introduced. Then, based on the illustration and hypothesis for the command agent’s cognition problem, the parameter’s normalization, the discrete of the Q table based on GRBF neural network, the strip temporal difference mechanism and the learning process of the structure of the network are explained for the improved Qleaning cognition algorithm. Finally, the validation and effectiveness of the algorithm is proved through the battle simulation experiment of the airground unify confrontation SoS. Besides, through a lot of visualization recall analysis for the C2 algorithm, we found that the coordination of the firepower and the continuous tactical maneuver are important to the operational effectiveness and injure decrease.
 

Key words: weapon system-of-systems, battle simulation, reinforcement learning, GRBF neural network, cognition and decision