Adaptive Quantization Based on Ensemble Distillation to Support FL Enabled Edge Intelligence

Liu, Y.-J., Qin, S., Feng, G., Niyato, D., Sun, Y. and Zhou, J. (2022) Adaptive Quantization Based on Ensemble Distillation to Support FL Enabled Edge Intelligence. In: 2022 IEEE Global Communications Conference (GLOBECOM), Rio de Janeiro, Brazil, 04-08 Dec 2022, pp. 2194-2199. ISBN 9781665435406 (doi: 10.1109/GLOBECOM48099.2022.10001182)

Abstract

Federated learning (FL) has recently become one of the most acknowledged technologies in promoting the development of intelligent edge networks with the ever-increasing computing capability of user equipment (UE). In traditional FL paradigm, local models are usually required to be homogeneous for aggregation to achieve an accurate global model. Moreover, considerable communication cost and training time may be incurred in resource-constrained edge networks due to a large number of UEs participating in model transmission and the large size of transmitted models. Therefore, it is imperative to develop effective training schemes for heterogeneous FL models, while reducing communication cost as well as training time. In this paper, we propose an adaptive quantization scheme based on ensemble distillation (AQeD) for FL to facilitate personalized quantized model training over heterogeneous local models with different size, structure, and quantization level, etc. Specifically, we design an augmented loss function by jointly considering distillation loss function, quantization values and available wireless resources, where UEs train their local personalized machine learning models and send the quantized models to a server. Based on local quantized models, the server first performs global aggregation for cluster ensembles and then sends the aggregated model of the cluster back to the participating UEs. Numerical results show that our proposed AQeD scheme can significantly reduce communication cost as well as training time in comparison with some known state-of-the-art solutions.

• Item Type: Conference Proceedings
• Additional Information: This work was supported by the Key Research and Development Projects under No. 2020YFB1806804, the National Natural Science Foundation under No. 62071091, the Huawei Cooperation Project under No. TC20210316002, and the China Scholarship Council under No. 202106070116.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Sun, Dr Yao and Feng, Professor Gang
• Authors: Liu, Y.-J., Qin, S., Feng, G., Niyato, D., Sun, Y., and Zhou, J.
• College/School: College of Science and Engineering > School of Engineering; College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity
• ISBN: 9781665435406
• Published Online: 11 January 2023
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in 2022 IEEE Global Communications Conference (GLOBECOM): 2194-2199
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation