Li, H., Shrestha, A., Heidari, H. , Le Kernec, J. and Fioranelli, F. (2020) Bi-LSTM network for multimodal continuous human activity recognition and fall detection. IEEE Sensors Journal, 20(3), pp. 1191-1201. (doi: 10.1109/JSEN.2019.2946095)
|
Text
199184.pdf - Accepted Version 2MB |
Abstract
This paper presents a framework based on multi-layer bi-LSTM network (bidirectional Long Short-Term Memory) for multimodal sensor fusion to sense and classify daily activities’ patterns and high-risk events such as falls. The data collected in this work are continuous activity streams from FMCW radar and three wearable inertial sensors on the wrist, waist, and ankle. Each activity has a variable duration in the data stream so that the transitions between activities can happen at random times within the stream, without resorting to conventional fixed-duration snapshots. The proposed bi-LSTM implements soft feature fusion between wearable sensors and radar data, as well as two robust hard-fusion methods using the confusion matrices of both sensors. A novel hybrid fusion scheme is then proposed to combine soft and hard fusion to push the classification performances to approximately 96% accuracy in identifying continuous activities and fall events. These fusion schemes implemented with the proposed bi-LSTM network are compared with conventional sliding window approach, and all are validated with realistic “leaving one participant out” (L1PO) method (i.e. testing subjects unknown to the classifier). The developed hybrid-fusion approach is capable of stabilizing the classification performance among different participants in terms of reducing accuracy variance of up to 18.1% and increasing minimum, worst-case accuracy up to 16.2%.
| Item Type: | Articles |
|---|---|
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Fioranelli, Dr Francesco and Heidari, Professor Hadi and Le Kernec, Dr Julien and Shrestha, Mr Aman and Li, Haobo |
| Authors: | Li, H., Shrestha, A., Heidari, H., Le Kernec, J., and Fioranelli, F. |
| College/School: | College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering College of Science and Engineering > School of Engineering > Systems Power and Energy |
| Journal Name: | IEEE Sensors Journal |
| Publisher: | IEEE |
| ISSN: | 1530-437X |
| ISSN (Online): | 1558-1748 |
| Published Online: | 07 October 2019 |
| Copyright Holders: | Copyright © 2019 Crown Copyright |
| First Published: | First published in IEEE Sensors Journal 20(3):1191-1201 |
| Publisher Policy: | Reproduced in accordance with the copyright policy of the publisher |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information