FedAdapt: Adaptive offloading for IoT Devices in federated learning

Wu, D., Ullah, R., Harvey, P. , Kilpatrick, P., Spence, I. and Varghese, B. (2022) FedAdapt: Adaptive offloading for IoT Devices in federated learning. IEEE Internet of Things Journal, 9(21), pp. 20889-20901. (doi: 10.1109/JIOT.2022.3176469)

Abstract

Applying federated learning (FL) on Internet of Things (IoT) devices is necessitated by the large volumes of data they produce and growing concerns of data privacy. However, there are three challenges that need to be addressed to make FL efficient: 1) execution on devices with limited computational capabilities; 2) accounting for stragglers due to computational heterogeneity of devices; and 3) adaptation to the changing network bandwidths. This article presents FedAdapt , an adaptive offloading FL framework to mitigate the aforementioned challenges. FedAdapt accelerates local training in computationally constrained devices by leveraging layer offloading of deep neural networks (DNNs) to servers. Furthermore, FedAdapt adopts reinforcement learning (RL)-based optimization and clustering to adaptively identify which layers of the DNN should be offloaded for each individual device on to a server to tackle the challenges of computational heterogeneity and changing network bandwidth. The experimental studies are carried out on a lab-based testbed and it is demonstrated that by offloading a DNN from the device to the server FedAdapt reduces the training time of a typical IoT device by over half compared to classic FL. The training time of extreme stragglers and the overall training time can be reduced by up to 57%. Furthermore, with changing network bandwidth, FedAdapt is demonstrated to reduce the training time by up to 40% when compared to classic FL, without sacrificing accuracy.

• Item Type: Articles
• Additional Information: This work was supported by Rakuten Mobile, Japan. The work of Blesson Varghese was supported by the Royal Society Short Industry Fellowship.
• Keywords: Training, servers, internet of things, computational modelling, bandwidth, data models, adaptation models, edge computing, federated learning (FL), Internet of Things (IoT), reinforcement learning (RL).
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Harvey, Dr Paul
• Authors: Wu, D., Ullah, R., Harvey, P., Kilpatrick, P., Spence, I., and Varghese, B.
• College/School: College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
• Journal Name: IEEE Internet of Things Journal
• Publisher: IEEE
• ISSN: 2327-4662
• ISSN (Online): 2327-4662
• Published Online: 19 May 2022
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in IEEE Internet of Things Journal 9(21):20889-20901
• Publisher Policy: Reproduced in accordance with the publisher copyright policy