TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users

Kulkarni, H., MacAvaney, S. , Goharian, N. and Frieder, O. (2022) TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users. In: 13th Language Resources and Evaluation Conference (LREC 2022), Marseille, France, 20-25 June 2022, pp. 2157-2165. ISBN 9791095546726

[img] Text
268533.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial.

547kB

Publisher's URL: http://www.lrec-conf.org/proceedings/lrec2022/index.html

Abstract

Social media are heavily used by many users to share their mental health concerns and diagnoses. This trend has turned social media into a large-scale resource for researchers focused on detecting mental health conditions. Social media usage varies considerably across individuals. Thus, classification of patterns, including detecting signs of depression, must account for such variation. We address the disparity in classification effectiveness for users with little activity (e.g., new users). Our evaluation, performed on a large-scale dataset, shows considerable detection discrepancy based on user posting frequency. For instance, the F1 detection score of users with an above-median versus below-median number of posts is greater than double (0.803 vs 0.365) using a conventional CNN-based model; similar results were observed on lexical and transformer-based classifiers. To complement this evaluation, we propose a dynamic thresholding technique that adjusts the classifier’s sensitivity as a function of the number of posts a user has. This technique alone reduces the margin between users with many and few posts, on average, by 45% across all methods and increases overall performance, on average, by 33%. These findings emphasize the importance of evaluating and tuning natural language systems for potentially vulnerable populations

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:MacAvaney, Dr Sean
Authors: Kulkarni, H., MacAvaney, S., Goharian, N., and Frieder, O.
College/School:College of Science and Engineering > School of Computing Science
ISBN:9791095546726
Copyright Holders:Copyright © European Language Resources Association (ELRA) 2022
First Published:First published in Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022): 2157-2165
Publisher Policy:Reproduced under a Creative Commons License
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record