Intelligent handover algorithm for vehicle-to-network communications with double-deep Q-learning

Tan, K. , Bremner, D. , Le Kernec, J. , Sambo, Y. , Zhang, L. and Imran, M. A. (2022) Intelligent handover algorithm for vehicle-to-network communications with double-deep Q-learning. IEEE Transactions on Vehicular Technology, 71(7), pp. 7848-7862. (doi: 10.1109/TVT.2022.3169804)

Abstract

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (UK) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72\% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller.

• Item Type: Articles
• Additional Information: This work was supported by the U.K. EPSRC (EP/S02476X/1).
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Zhang, Professor Lei and Tan, Kang and Imran, Professor Muhammad and Sambo, Dr Yusuf and Le Kernec, Dr Julien and Bremner, Dr Duncan
• Authors: Tan, K., Bremner, D., Le Kernec, J., Sambo, Y., Zhang, L., and Imran, M. A.
• College/School: College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity; College of Science and Engineering > School of Engineering > Systems Power and Energy
• Journal Name: IEEE Transactions on Vehicular Technology
• Publisher: IEEE
• ISSN: 0018-9545
• ISSN (Online): 1939-9359
• Published Online: 25 April 2022
• Copyright Holders: Copyright © 2022 The Authors
• First Published: First published in IEEE Transactions on Vehicular Technology 71(7): 7848-7862
• Publisher Policy: Reproduced under a Creative Commons license

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
304481		Resource Orchestration for Diverse Radio Systems	Lei Zhang	Engineering and Physical Sciences Research Council (EPSRC)	EP/S02476X/1	ENG - Systems Power & Energy