In a one-on-one air combat game, the opponentâ??s maneuver strategy is usually not deterministic, which leads us to consider a\nvariety of opponentâ??s strategies when designing our maneuver strategy. In this paper, an alternate freeze game framework based on\ndeep reinforcement learning is proposed to generate the maneuver strategy in an air combat pursuit. Themaneuver strategy agents\nfor aircraft guidance of both sides are designed in a flight level with fixed velocity and the one-on-one air combat scenario.\nMiddleware which connects the agents and air combat simulation software is developed to provide a reinforcement learning\nenvironment for agent training. A reward shaping approach is used, by which the training speed is increased, and the performance\nof the generated trajectory is improved. Agents are trained by alternate freeze games with a deep reinforcement algorithm to deal\nwith nonstationarity. A league system is adopted to avoid the red queen effect in the game where both sides implement adaptive\nstrategies. Simulation results show that the proposed approach can be applied to maneuver guidance in air combat, and typical\nangle fight tactics can be learnt by the deep reinforcement learning agents. For the training of an opponent with the adaptive\nstrategy, the winning rate can reach more than 50%, and the losing rate can be reduced to less than 15%. In a competition with all\nopponents, the winning rate of the strategic agent selected by the league system is more than 44%, and the probability of not losing\nis about 75%.
Loading....