Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning

Chen, T. , Bu, S. , Liu, X., Kang, J., Yu, F. R. and Han, Z. (2022) Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning. IEEE Transactions on Smart Grid, 13(1), p. 715. (doi: 10.1109/TSG.2021.3124465)

Abstract

A key aspect of multi-energy microgrids (MEMGs) is the capability to efficiently convert and store energy in order to reduce the costs and environmental impact. Peer-to-peer (P2P) energy trading is a novel paradigm for decentralised energy market designs. In this paper, we investigate the external P2P energy trading problem and internal energy conversion problem within interconnected residential, commercial and industrial MEMGs. These two problems are complex decision-making problems with enormous high-dimensional data and uncertainty, so a multi-agent deep reinforcement learning approach combining the multi-agent actor-critic algorithm with the twin delayed deep deterministic policy gradient algorithm is proposed. The proposed approach can handle the high-dimensional continuous action space and aligns with the nature of P2P energy trading with multiple MEMGs. Simulation results based on three real-world MG datasets show that the proposed approach significantly reduces each MG’s average hourly operation cost. The impact of carbon tax pricing is also considered.

• Item Type: Articles
• Additional Information: This work was supported by start-up funds provided by Brock University, University of Glasgow Principal’s Early Career Mobility fund, NSF CNS2128368, CNS-2107216, Toyota and Amazon.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Chen, Tianyi and Bu, Dr Shengrong
• Authors: Chen, T., Bu, S., Liu, X., Kang, J., Yu, F. R., and Han, Z.
• College/School: College of Science and Engineering > School of Engineering > Systems Power and Energy
• Journal Name: IEEE Transactions on Smart Grid
• Publisher: IEEE
• ISSN: 1949-3053
• ISSN (Online): 1949-3053
• Published Online: 01 November 2021
• Copyright Holders: Copyright © 2021 IEEE
• First Published: First published in IEEE Transactions on Smart Grid 13(1): 715-727
• Publisher Policy: Reproduced in accordance with the publisher copyright policy