Recognizing British sign language using deep learning: a contactless and privacy-preserving approach

Hameed, H., Usman, M., Tahir, A., Ahmad, K., Hussain, A., Imran, M. A. and Abbasi, Q. H. (2023) Recognizing British sign language using deep learning: a contactless and privacy-preserving approach. IEEE Transactions on Computational Social Systems, 10(4), pp. 2090-2098. (doi: 10.1109/TCSS.2022.3210288)

Abstract

Sign language is utilized by deaf-mute to communicate through hand movements, body postures, and facial emotions. The motions in sign language comprise a range of distinct hand and finger articulations that are occasionally synchronized with the head, face, and body. Automatic sign language recognition (SLR) is a highly challenging area and still remains in its infancy compared with speech recognition after almost three decades of research. Current wearable and vision-based systems for SLR are intrusive and suffer from the limitations of ambient lighting and privacy concerns. To the best of our knowledge, our work proposes the first contactless British sign language (BSL) recognition system using radar and deep learning (DL) algorithms. Our proposed system extracts the 2-D spatiotemporal features from the radar data and applies the state-of-the-art DL models to classify spatiotemporal features from BSL signs to different verbs and emotions, such as Help, Drink, Eat, Happy, Hate, and Sad. We collected and annotated a large-scale benchmark BSL dataset covering 15 different types of BSL signs. Our proposed system demonstrates highest classification performance with a multiclass accuracy of up to 90.07% at a distance of 141 cm from the subject using the VGGNet model.

• Item Type: Articles
• Additional Information: This work was supported in parts by Engineering and Physical Sciences Research Council (EPSRC) grants EP/T021020/1 and EP/T021063/1.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Tahir, Dr Ahsen and Hameed, Mrs Hira and Imran, Professor Muhammad and Usman, Dr Muhammad and Abbasi, Professor Qammer
• Authors: Hameed, H., Usman, M., Tahir, A., Ahmad, K., Hussain, A., Imran, M. A., and Abbasi, Q. H.
• College/School: College of Science and Engineering > School of Engineering; College of Science and Engineering > School of Engineering > Autonomous Systems and Connectivity; College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
• Journal Name: IEEE Transactions on Computational Social Systems
• Publisher: IEEE
• ISSN: 2329-924X
• ISSN (Online): 2329-924X
• Published Online: 13 October 2022
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in IEEE Transactions on Computational Social Systems 10(4):2090-2098
• Publisher Policy: Reproduced with the permission of the publisher

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
307829		Quantum-Inspired Imaging for Remote Monitoring of Health & Disease in Community Healthcare	Jonathan Cooper	Engineering and Physical Sciences Research Council (EPSRC)	EP/T021020/1	ENG - Biomedical Engineering