A novel handover scheme for millimeter wave network: an approach of integrating reinforcement learning and optimization

Wang, R., Sun, Y. , Zhang, C., Yang, B., Imran, M. and Zhang, L. (2023) A novel handover scheme for millimeter wave network: an approach of integrating reinforcement learning and optimization. Digital Communications and Networks, (doi: 10.1016/j.dcan.2023.08.002) (In Press)

Abstract

The millimeter-Wave (mmWave) communication with the advantages of abundant bandwidth and immunity to interference has been deemed a promising technology to greatly improve network capacity. However, due to such characteristics of mmWave, as short transmission distance, high sensitivity to the blockage, and large propagation path loss, handover issues (including trigger condition and target beam selection) become much complicated. In this paper, we design a novel handover scheme to optimize the overall system throughput as well as the total system delay while guaranteeing the Quality of Service (QoS) of each User Equipment (UE). Specifically, the proposed handover scheme called O-MAPPO integrates the Reinforcement Learning (RL) algorithm and optimization theory. The RL algorithm known as Multi-Agent Proximal Policy Optimization (MAPPO) plays a role in determining handover trigger conditions. Further, we propose an optimization problem in conjunction with MAPPO to select the target base station. The aim is to evaluate and optimize the system performance of total throughput and delay while guaranteeing the QoS of each UE after the handover decision is made. The numerical results show the overall system throughput and delay with our method are slightly worse than that with the exhaustive search method but much better than that using another typical RL algorithm Deep Deterministic Policy Gradient (DDPG).

• Item Type: Articles
• Keywords: Reinforcement learning, optimization theory, MmWave, cellular network, handover.
• Status: In Press
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Yang, Bowen and Zhang, Chao and Zhang, Professor Lei and Imran, Professor Muhammad and Wang, Mr Ruiyu and Sun, Dr Yao
• Authors: Wang, R., Sun, Y., Zhang, C., Yang, B., Imran, M., and Zhang, L.
• College/School: College of Science and Engineering; College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity
• Journal Name: Digital Communications and Networks
• Publisher: Elsevier
• ISSN: 2352-8648
• ISSN (Online): 2352-8648
• Published Online: 29 August 2023
• Copyright Holders: Copyright © 2023 Chongqing University of Posts and Telecommunications
• First Published: First published in Digital Communications and Networks 2023
• Publisher Policy: Reproduced under a Creative Commons license