The Impact of Face Mask and Emotion on Automatic Speech Recognition (ASR) and Speech Emotion Recognition (SER)

Oh, Q. Q., Seow, C. K. , Yusuff, M., Pranata, S. and Cao, Q. (2023) The Impact of Face Mask and Emotion on Automatic Speech Recognition (ASR) and Speech Emotion Recognition (SER). In: 2023 8th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, China, 26-28 Apr 2023, pp. 523-531. ISBN 9781665455336 (doi: 10.1109/ICCCBDA56900.2023.10154691)

Abstract

Meetings are common in the workplace, and it's critical to understand employees' feelings during a meeting since emotions influence how individuals talk and behave. Furthermore, due to Covid-19, individuals are required to wear a face mask in meetings, and there is not much research on how wearing a face mask impact the underlying emotion through speech. Although past research has shown that emotions and face masks have some form of impact on ASR, respectively, as of the time of writing, there are no findings on which attribute (emotion or mask) impacts the ASR system more and contributes more significantly to the speech processing error. Experiments were conducted, which aligned with the previous research that the face masks have a relatively small impact on ASR System performance at the sentence level. In addition, emotion affects speech significantly, especially sadness. Therefore, emotion contributes more significantly to speech processing error than a face mask. The experiments proved that the augmentation technique is useful as the F1 Score improved, with the greatest improvement being a 10% increase. The experiment also proved that using a face mask does not have much impact on emotion, making SER a feasible solution for detecting the underlying emotion through speech.

• Item Type: Conference Proceedings
• Keywords: Speech emotion, asr, ser, nlp.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Cao, Dr Qi and Seow, Dr Chee Kiat
• Authors: Oh, Q. Q., Seow, C. K., Yusuff, M., Pranata, S., and Cao, Q.
• Subjects: Q Science > Q Science (General); Q Science > QA Mathematics; Q Science > QA Mathematics > QA75 Electronic computers. Computer science; Q Science > QA Mathematics > QA76 Computer software; T Technology > T Technology (General); T Technology > TA Engineering (General). Civil engineering (General); T Technology > TK Electrical engineering. Electronics Nuclear engineering
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 2832-3734
• ISBN: 9781665455336
• Copyright Holders: Copyright © 2023 IEEE
• First Published: First published in 2023 8th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA)
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation
• Publisher