Proactive content caching based on actor-critic reinforcement learning for mobile edge networks

Jiang, W., Feng, D., Sun, Y. , Feng, G., Wang, Z. and Xia, X.-G. (2022) Proactive content caching based on actor-critic reinforcement learning for mobile edge networks. IEEE Transactions on Cognitive Communications and Networking, 8(2), pp. 1239-1252. (doi: 10.1109/TCCN.2021.3130995)

Abstract

Mobile edge caching/computing (MEC) has emerged as a promising approach for addressing the drastic increasing mobile data traffic by bringing high caching and computing capabilities to the edge of networks. Under MEC architecture, content providers (CPs) are allowed to lease some virtual machines (VMs) at MEC servers to proactively cache popular contents for improving users’ quality of experience. The scalable cache resource model rises the challenge for determining the ideal number of leased VMs for CPs to obtain the minimum expected downloading delay of users at the lowest caching cost. To address these challenges, in this paper, we propose an actor-critic (AC) reinforcement learning based proactive caching policy for mobile edge networks without the prior knowledge of users’ content demand. Specifically, we formulate the proactive caching problem under dynamical users’ content demand as a Markov decision process and propose a AC based caching algorithm to minimize the caching cost and the expected downloading delay. Particularly, to reduce the computational complexity, a branching neural network is employed to approximate the policy function in the actor part. Numerical results show that the proposed caching algorithm can significantly reduce the total cost and the average downloading delay when compared with other popular algorithms.

• Item Type: Articles
• Additional Information: This work was supported in part by the National Key R&D Program of China under Grant 2020YFB1807601, and in part by the Innovation Project of Guangdong Educational Department under Grant 2019KTSCX147, and in part by the Shenzhen Science and Technology Program under Grants JCYJ20210324095209025, and in part by ZTE Industry-Academia-Research Cooperation Funds.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Sun, Dr Yao and Feng, Professor Gang
• Authors: Jiang, W., Feng, D., Sun, Y., Feng, G., Wang, Z., and Xia, X.-G.
• College/School: College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity
• Journal Name: IEEE Transactions on Cognitive Communications and Networking
• Publisher: IEEE
• ISSN: 2332-7731
• ISSN (Online): 2332-7731
• Published Online: 26 November 2021
• Copyright Holders: Copyright © 2021 IEEE
• First Published: First published in IEEE Transactions on Cognitive Communications and Networking 8(2): 1239-1252
• Publisher Policy: Reproduced in accordance with the publisher copyright policy