Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition

Gu, X., Guo, Y., Deligianni, F. , Lo, B. and Yang, G.-Z. (2021) Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition. IEEE Transactions on Neural Networks and Learning Systems, 32(2), pp. 546-560. (doi: 10.1109/TNNLS.2020.3009448)

Abstract

For abnormal gait recognition, pattern-specific features indicating abnormalities are interleaved with the subject-specific differences representing biometric traits. Deep representations are, therefore, prone to overfitting, and the models derived cannot generalize well to new subjects. Furthermore, there is limited availability of abnormal gait data obtained from precise Motion Capture (Mocap) systems because of regulatory issues and slow adaptation of new technologies in health care. On the other hand, data captured from markerless vision sensors or wearable sensors can be obtained in home environments, but noises from such devices may prevent the effective extraction of relevant features. To address these challenges, we propose a cascade of deep architectures that can encode cross-modal and cross-subject transfer for abnormal gait recognition. Cross-modal transfer maps noisy data obtained from RGBD and wearable sensors to accurate 4-D representations of the lower limb and joints obtained from the Mocap system. Subsequently, cross-subject transfer allows disentangling subject-specific from abnormal pattern-specific gait features based on a multiencoder autoencoder architecture. To validate the proposed methodology, we obtained multimodal gait data based on a multicamera motion capture system along with synchronized recordings of electromyography (EMG) data and 4-D skeleton data extracted from a single RGBD camera. Classification accuracy was improved significantly in both Mocap and noisy modalities.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Deligianni, Dr Fani
• Authors: Gu, X., Guo, Y., Deligianni, F., Lo, B., and Yang, G.-Z.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: IEEE Transactions on Neural Networks and Learning Systems
• Publisher: IEEE
• ISSN: 2162-237X
• ISSN (Online): 2162-2388
• Published Online: 29 July 2020
• Copyright Holders: Copyright © 2020 IEEE
• First Published: First published in IEEE Transactions on Neural Networks and Learning Systems 32(2):546-560
• Publisher Policy: Reproduced in accordance with the publisher copyright policy