Alignbodynet: deep learning-based alignment of non-overlapping partial body point clouds from a single depth camera

Hu, P., Ho, E. S.L. and Munteanu, A. (2023) Alignbodynet: deep learning-based alignment of non-overlapping partial body point clouds from a single depth camera. IEEE Transactions on Instrumentation and Measurement, 72, 2502609. (doi: 10.1109/TIM.2022.3222501)

Abstract

This paper proposes a novel deep learning framework to generate omnidirectional 3D point clouds of human bodies by registering the front- and back-facing partial scans captured by a single depth camera. Our approach does not require calibration-assisting devices, canonical postures, nor does it make assumptions concerning an initial alignment or correspondences between the partial scans. This is achieved by factoring this challenging problem into ( i ) building virtual correspondences for partial scans, and ( ii ) implicitly predicting the rigid transformation between the two partial scans via the predicted virtual correspondences. In this study, we regress the SMPL vertices from the two partial scans for building the virtual correspondences. The main challenges are ( i ) estimating the body shape and pose under clothing from single partial dressed body point clouds, and ( ii ) the predicted bodies from front- and back-facing inputs required to be the same. We, thus, propose a novel deep neural network dubbed AlignBodyNet that introduces shape-interrelated features and a shape-constraint loss for resolving this problem.We also provide a simple yet efficient method for generating real-world partial scans from complete models, which fills the gap in the lack of quantitative comparisons based on the real-world data for various studies including partial registration, shape completion, and view synthesis. Experiments based on synthetic and real-world data show that our method achieves state-of-the-art performance in both objective and subjective terms.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Ho, Dr Edmond S. L
• Authors: Hu, P., Ho, E. S.L., and Munteanu, A.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: IEEE Transactions on Instrumentation and Measurement
• Publisher: IEEE
• ISSN: 0018-9456
• ISSN (Online): 1557-9662
• Published Online: 16 November 2022
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in IEEE Transactions on Instrumentation and Measurement 72: 2502609
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher