An intelligent implementation of multi-sensing data fusion with neuromorphic computing for human activity recognition

Yu, Z., Zahid, A., Taha, A. , Taylor, W. , Le Kernec, J. , Heidari, H. , Imran, M. A. and Abbasi, Q. H. (2023) An intelligent implementation of multi-sensing data fusion with neuromorphic computing for human activity recognition. IEEE Internet of Things Journal, 10(2), pp. 1124-1133. (doi: 10.1109/JIOT.2022.3204581)

Abstract

The increasing demand for considering multi-sensor data fusion technology has drawn attention for precise human activity recognition over standalone technology due to its reliability and robustness. This paper presents a framework that fuses data from multiple sensing systems and applies Neuromorphic computing to sense and classify human activities. The data is collected by utilizing Inertial Measurement Unit (IMU) sensors, software-defined radios, and radars and feature extraction and selection are performed on the data. For each of the actions, such as sitting and standing, an activity matrix is generated, which is then fed into a discrete Hopfield neural network as a binary feature pattern for one-shot learning. Following the Hopfield network neurons’ feedback output, the conformity to the standard activity feature pattern is also determined. Following the Hopfield network neurons’ feedback output, the training of neurons is completed after 2 steps under the Hebbian learning law, and the conformity to the standard activity feature pattern is also determined. According to probabilistic statistics on inference predictions, the proposed method that Neuromorphic computing of the three data fused framework achieved the Box-plot for highest lower quartile output of 95.34%, while the confusion matrix classification accuracy of the two activities was 98.98%. The results have shown that Neuromorphic computing is most capable for multi-sensor data fusion-based human activity recognition. Furthermore, the proposed method can be enhanced by incorporating additional hardware signal processing in the system to enable the flexible integration of human activity data.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Taha, Dr Ahmad and Yu, Zheqi and Abbasi, Professor Qammer and Imran, Professor Muhammad and Zahid, Mr Adnan and Heidari, Professor Hadi and Taylor, William and Le Kernec, Dr Julien
• Authors: Yu, Z., Zahid, A., Taha, A., Taylor, W., Le Kernec, J., Heidari, H., Imran, M. A., and Abbasi, Q. H.
• College/School: College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity; College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
• Journal Name: IEEE Internet of Things Journal
• Publisher: IEEE
• ISSN: 2327-4662
• ISSN (Online): 2327-4662
• Published Online: 06 September 2022
• Copyright Holders: Copyright © 2022 The Authors
• First Published: First published in IEEE Internet of Things Journal 10(2):1124-1133
• Publisher Policy: Reproduced under a Creative Commons License

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
307829		Quantum-Inspired Imaging for Remote Monitoring of Health & Disease in Community Healthcare	Jonathan Cooper	Engineering and Physical Sciences Research Council (EPSRC)	EP/T021020/1	ENG - Biomedical Engineering