What you say or how you say it? Depression detection through joint modeling of linguistic and acoustic aspects of speech

Aloshban, N., Esposito, A. and Vinciarelli, A. (2021) What you say or how you say it? Depression detection through joint modeling of linguistic and acoustic aspects of speech. Cognitive Computation, (doi: 10.1007/s12559-020-09808-3) (Early Online Publication)

[img] Text
233197.pdf - Published Version
Available under License Creative Commons Attribution.

1MB

Abstract

Depression is one of the most common mental health issues. (It affects more than 4% of the world’s population, according to recent estimates.) This article shows that the joint analysis of linguistic and acoustic aspects of speech allows one to discriminate between depressed and nondepressed speakers with an accuracy above 80%. The approach used in the work is based on networks designed for sequence modeling (bidirectional Long-Short Term Memory networks) and multimodal analysis methodologies (late fusion, joint representation and gated multimodal units). The experiments were performed over a corpus of 59 interviews (roughly 4 hours of material) involving 29 individuals diagnosed with depression and 30 control participants. In addition to an accuracy of 80%, the results show that multimodal approaches perform better than unimodal ones owing to people’s tendency to manifest their condition through one modality only, a source of diversity across unimodal approaches. In addition, the experiments show that it is possible to measure the “confidence” of the approach and automatically identify a subset of the test data in which the performance is above a predefined threshold. It is possible to effectively detect depression by using unobtrusive and inexpensive technologies based on the automatic analysis of speech and language.

Item Type:Articles
Status:Early Online Publication
Refereed:Yes
Glasgow Author(s) Enlighten ID:ALOSHBAN, NUJUD IBRAHIM Z and Vinciarelli, Professor Alessandro
Authors: Aloshban, N., Esposito, A., and Vinciarelli, A.
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Cognitive Computation
Publisher:Springer
ISSN:1866-9956
ISSN (Online):1866-9964
Published Online:24 February 2021
Copyright Holders:Copyright © The Author(s) 2021
First Published:First published in Cognitive Computation 2021
Publisher Policy:Reproduced under a Creative Commons license

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
303764EPSRC CDT - Socially Intelligent Artificial AgentsAlessandro VinciarelliEngineering and Physical Sciences Research Council (EPSRC)EP/S02266X/1Computing Science
172889A Robot Training Buddy for adults with ASDAlessandro VinciarelliEngineering and Physical Sciences Research Council (EPSRC)EP/N035305/1Computing Science