Wireless federated Langevin Monte Carlo: repurposing channel noise for Bayesian sampling and privacy

Liu, D. and Simeone, O. (2023) Wireless federated Langevin Monte Carlo: repurposing channel noise for Bayesian sampling and privacy. IEEE Transactions on Wireless Communications, 22(5), pp. 2946-2961. (doi: 10.1109/TWC.2022.3215663)

Abstract

Most works on federated learning (FL) focus on the most common frequentist formulation of learning whereby the goal is minimizing the global empirical loss. Frequentist learning, however, is known to be problematic in the regime of limited data as it fails to quantify epistemic uncertainty in prediction. Bayesian learning provides a principled solution to this problem by shifting the optimization domain to the space of distribution in the model parameters. This paper proposes a novel mechanism for the efficient implementation of Bayesian learning in wireless systems. Specifically, we focus on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless Federated LMC (WFLMC), that is able to repurpose channel noise for the double role of seed randomness for MCMC sampling and of privacy preservation. To this end, based on the analysis of the Wasserstein distance between sample distribution and global posterior distribution under privacy and power constraints, we introduce a power allocation strategy as the solution of a convex program. The analysis identifies distinct operating regimes in which the performance of the system is power-limited, privacy-limited, or limited by the requirement of MCMC sampling. Both analytical and simulation results demonstrate that, if the channel noise is properly accounted for under suitable conditions, it can be fully repurposed for both MCMC sampling and privacy preservation, obtaining the same performance as in an ideal communication setting that is not subject to privacy constraints.

• Item Type: Articles
• Additional Information: The work of O. Simeone was supported by European Research Council (ERC) under the European Unions Horizon 2020 Research and Innovation Programme (Grant Agreement No. 725731) and an Open Fellowship by the EPSRC.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Liu, Dr Dongzhu
• Authors: Liu, D., and Simeone, O.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: IEEE Transactions on Wireless Communications
• Publisher: IEEE
• ISSN: 1536-1276
• ISSN (Online): 1558-2248
• Published Online: 26 October 2022
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in IEEE Transactions on Wireless Communications 22(5): 2946-2961
• Publisher Policy: Reproduced in accordance with the publisher copyright policy