CL3D: Camera-LiDAR 3D object detection with point feature enhancement and point-guided fusion

Lin, C., Tian, D., Duan, X., Zhou, J., Zhao, D. and Cao, D. (2022) CL3D: Camera-LiDAR 3D object detection with point feature enhancement and point-guided fusion. IEEE Transactions on Intelligent Transportation Systems, 23(10), pp. 18040-18050. (doi: 10.1109/TITS.2022.3154537)

[img] Text
266475.pdf - Accepted Version

3MB

Abstract

Camera-LiDAR 3D object detection has been extensively investigated due to its significance for many real-world applications. However, there are still of great challenges to address the intrinsic data difference and perform accurate feature fusion among two modalities. To these ends, we propose a two-stream architecture termed as CL3D, that integrates with point enhancement module, point-guided fusion module with IoU-aware head for cross-modal 3D object detection. Specifically, pseudo LiDAR is firstly generated from RGB image, and point enhancement module (PEM) is then designed to enhance the raw LiDAR with pseudo point. Moreover, point-guided fusion module (PFM) is developed to find image-point correspondence at different resolutions, and incorporate semantic with geometric features in a point-wise manner. We also investigate the inconsistency between localization confidence and classification score in 3D detection, and introduce IoU-aware prediction head (IoU Head) for accurate box regression. Comprehensive experiments are conducted on publicly available KITTI dataset, and CL3D reports the outstanding detection performance compared to both single- and multi-modal 3D detectors, demonstrating its effectiveness and competitiveness.

Item Type:Articles
Additional Information:This work was supported in part by the National Natural Science Foundation of China under Grant 62061130221, Grant U20A20155, and Grant 62173012; in part by the Beijing Municipal Natural Science Foundation under Grant L191001; in part by the Zhuoyue Program of Beihang University (Postdoctoral Fellowship); and in part by the China Postdoctoral Science Foundation under Grant 2020M680299.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Zhao, Dr Dezong
Authors: Lin, C., Tian, D., Duan, X., Zhou, J., Zhao, D., and Cao, D.
College/School:College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity
Journal Name:IEEE Transactions on Intelligent Transportation Systems
Publisher:IEEE
ISSN:1524-9050
ISSN (Online):1558-0016
Published Online:03 March 2022
Copyright Holders:Copyright © 2022 IEEE
First Published:First published in IEEE Transactions on Intelligent Transportation Systems 23(10): 18040-18050
Publisher Policy:Reproduced in accordance with the publisher copyright policy

University Staff: Request a correction | Enlighten Editors: Update this record