• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2021, Vol. 43 ›› Issue (05): 836-844.

Previous Articles     Next Articles

A multi-agent Q-learning based selection method for heterogeneous vehicular network

NIE Lei1,2,LIU Bo1,2,LI Peng1,2,HE Heng1,2   

  1. (1.School of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430065;

    2.Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System,

    Wuhan University of Science and Technology,Wuhan 430065,China)

  • Received:2020-11-20 Revised:2021-01-06 Accepted:2021-05-25 Online:2021-05-25 Published:2021-05-19

Abstract: How to select an access network in heterogeneous vehicular network environment is crucial for the service experience of vehicular terminal users. The current Q-learning based network selection method uses the interaction between the agent and the environment to iteratively learn network selection strategies and further realize better network resource allocation. However, this kind of methods usually have the problems of inefficient iterations and slow convergence caused by oversized state space. Besides, overestimations caused by the updates of Q tables lead to unreasonable utilization of network resources. Aiming at above problems, a Multi-agent Q-learning based Selection Method (MQSM) is proposed for heterogeneous vehicular network with 5G communication. The above method adopts the multi-agent cooperative learning idea and gets the total return value of action selection by alternate update of double Q tables. Finally, it achieves a long-term effective optimal network selection decision set in heterogeneous vehicular network environment. Experiment results show that, compared with similar methods, MQSM has better performance in terms of total system handovers, average discount values and network resource utilization.


Key words: multi-agent, Q-learning, network selection, heterogeneous vehicular network, 5G communication