Deep active cross-modal visuo-tactile transfer learning for robotic object recognition

Murali, P. K. , Wang, C., Lee, D., Dahiya, R. and Kaboli, M. (2022) Deep active cross-modal visuo-tactile transfer learning for robotic object recognition. IEEE Robotics and Automation Letters, 7(4), 9557 - 9564. (doi: 10.1109/LRA.2022.3191408)

Abstract

We propose for the first time, a novel deep active visuo-tactile cross-modal full-fledged framework for object recognition by autonomous robotic systems. Our proposed network xAVTNet is actively trained with labelled point clouds from a vision sensor with one robot and tested with an active tactile perception strategy to recognise objects never touched before using another robot. We propose a novel visuo-tactile loss (VTLoss) to minimise the discrepancy between the visual and tactile domains for unsupervised domain adaptation. Our framework leverages the strengths of deep neural networks for cross-modal recognition along with active perception and active learning strategies for increased efficiency by minimising redundant data collection. Our method is extensively evaluated on a real robotic system and compared against baselines and other state-of-art approaches. We demonstrate clear outperformance in recognition accuracy compared to the state-of-art visuo-tactile cross-modal recognition method.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Dahiya, Professor Ravinder and Murali, Prajval Kumar
• Authors: Murali, P. K., Wang, C., Lee, D., Dahiya, R., and Kaboli, M.
• College/School: College of Science and Engineering; College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
• Journal Name: IEEE Robotics and Automation Letters
• Publisher: IEEE
• ISSN: 2377-3774
• ISSN (Online): 2377-3766
• Published Online: 15 July 2022
• Copyright Holders: Copyright © 2022 The Authors
• First Published: First published in IEEE Robotics and Automation Letters 7(4):9557-9564
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher