AI-enabled soft sensing array for simultaneous detection of muscle deformation and mechanomyography for metaverse somatosensory interaction

Suo, J. et al. (2024) AI-enabled soft sensing array for simultaneous detection of muscle deformation and mechanomyography for metaverse somatosensory interaction. Advanced Science, 11(16), 2305025. (doi: 10.1002/advs.202305025) (PMID:38376001)

Abstract

Motion recognition (MR)-based somatosensory interaction technology, which interprets user movements as input instructions, presents a natural approach for promoting human-computer interaction, a critical element for advancing metaverse applications. Herein, this work introduces a non-intrusive muscle-sensing wearable device, that in conjunction with machine learning, enables motion-control-based somatosensory interaction with metaverse avatars. To facilitate MR, the proposed device simultaneously detects muscle mechanical activities, including dynamic muscle shape changes and vibrational mechanomyogram signals, utilizing a flexible 16-channel pressure sensor array (weighing ≈0.38 g). Leveraging the rich information from multiple channels, a recognition accuracy of ≈96.06% is achieved by classifying ten lower-limb motions executed by ten human subjects. In addition, this work demonstrates the practical application of muscle-sensing-based somatosensory interaction, using the proposed wearable device, for enabling the real-time control of avatars in a virtual space. This study provides an alternative approach to traditional rigid inertial measurement units and electromyography-based methods for achieving accurate human motion capture, which can further broaden the applications of motion-interactive wearable devices for the coming metaverse age.

• Item Type: Articles
• Additional Information: This work was partially supported by funding from the National Natural Science Foundation of China (Project No. 62073208) and the Hong Kong Research Grants Council: 1) the Theme-based Research Scheme Project No. T42-717/20-R, 2) the General Research Fund Project No. 11210819,and 3) the Collaborative Research Fund Project No. C7174-20G.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Vellaisamy, Professor Roy
• Authors: Suo, J., Liu, Y., Wang, J., Chen, M., Wang, K., Yang, X., Yao, K., Vellaisamy, A.L. R., Yu, X., Daoud, W. A., Liu, N., Wang, J., Wang, Z., and Li, W. J.
• College/School: College of Science and Engineering > School of Engineering
• Journal Name: Advanced Science
• Publisher: Wiley
• ISSN: 2198-3844
• ISSN (Online): 2198-3844
• Published Online: 20 February 2024
• Copyright Holders: Copyright: © 2023 The Authors
• First Published: First published in Advanced Science 11(16): 2305025
• Publisher Policy: Reproduced under a Creative Commons licence