Multi-scale rain removal across multiple frequencies sub-bands using MS-CycleGANs

Tang, L. M. , Lim, L. H. I. and Siebert, P. (2023) Multi-scale rain removal across multiple frequencies sub-bands using MS-CycleGANs. In: Carbone, G., Laribi, M. A. and Jiang, Z. (eds.) Advances in Automation, Mechanical and Design Engineering. Series: Mechanisms and machine science. Springer: Cham, pp. 233-245. ISBN 9783031400728 (doi: 10.1007/978-3-031-40070-4_18)

Abstract

Real rain removal is a complex and challenging problem on real rain images. Although rain removal using the CycleGAN (Tang et al. in ECCV 2018 Workshop Proceedings, Part V [1]) was found to be superior as compared to other existing techniques, there are two aspects that can be improved on: (1) Preserve original scene details, and (2) Remove low-frequency rain distortion. Hence, this paper proposes a multi-scale representation technique, called the MS-CycleGANs, to address the remaining gaps of the CycleGAN in rain removal. It uses a pyramid framework that is made up of multi-scale CycleGANs. Such multi-scale representation is capable of removing rain distortions at multiple frequency sub-bands without sacrificing the scene details because the characteristics of both rain and no-rain domains can be learnt simultaneously by the proposed network at their frequency sub-bands. We compare the MS-CycleGANs to the CycleGAN on similar rain images for the networks’ training and testing. The results are compared quantitatively using the NIQE metric (Mittal et al. in IEEE Signal Process Lett 22:209–212, [2]) in this paper as we do not have any prior information on real rain distortions. The results have demonstrated the MS-CycleGANs’ superiority in removing real rain distortions, both qualitatively by visual check and also quantitatively using the NIQE metric (Mittal et al. in IEEE Signal Process Lett 22:209–212, [2]).

• Item Type: Book Sections
• Status: Published
• Glasgow Author(s) Enlighten ID: Tang, Dr Lai Meng and Siebert, Dr Paul and Lim, Dr Li Hong Idris
• Authors: Tang, L. M., Lim, L. H. I., and Siebert, P.
• College/School: College of Science and Engineering; College of Science and Engineering > School of Computing Science
• Publisher: Springer
• ISBN: 9783031400728