How reinforcement learning is helping to solve internet-of-underwater-things problems

Omeke, K. G., Abubakar, A. I. , Zhang, L. , Abbasi, Q. H. and Imran, M. A. (2022) How reinforcement learning is helping to solve internet-of-underwater-things problems. IEEE Internet of Things Magazine, 5(4), pp. 24-29. (doi: 10.1109/IOTM.001.2200129)

Abstract

We present a review of how reinforcement learning (RL) is helping to tackle some of the most challenging problems in the Internet-of-underwater-things (IoUTs). Scientists estimate that 50-80 percent of atmospheric oxygen comes from the ocean, implying that life on earth depends heavily on clean and healthy oceans. This huge significance of the ocean in supporting life on earth is motivating the use of artificial intelligence (AI) and machine learning (ML) tools to create a sustainable marine ecosystem. We briefly review the RL paradigm, its categorisations and RL algorithms developed to solve important problems in IoUTs. New literature keeps emerging that show innovative applications of RL in underwater communications and networking that far outperform conventional solutions and other ML-based methods. Due to its online learning nature, RL is particularly useful for decision making in dynamic environments such as underwater where the communication channel is stochastic and rapidly varying. We explore the applications of RL in IoUTs, showing different classes of IoUTs problems and high-lighting RL algorithms that are tailored to solving them. Despite the significant progress that has made in the RL field, there are still many challenges and open research problems in the use of RL in IoUTs. We conclude the article with an outline of some of these challenges and suggest some ways forward.

• Item Type: Articles
• Additional Information: This work was funded by the Petroleum Technology Development Fund (PTDF) of the Federal Republic of Nigeria.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Zhang, Professor Lei and Abubakar, Mr Attai and Imran, Professor Muhammad and Omeke, Dr Kenechi and Abbasi, Professor Qammer
• Authors: Omeke, K. G., Abubakar, A. I., Zhang, L., Abbasi, Q. H., and Imran, M. A.
• College/School: College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity; College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
• Journal Name: IEEE Internet of Things Magazine
• Publisher: IEEE
• ISSN: 2576-3180
• ISSN (Online): 2576-3199
• Published Online: 11 January 2023
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in IEEE Internet of Things Magazine 5(4): 24-29
• Publisher Policy: Reproduced in accordance with the publisher copyright policy