Introducing the 3MT_French dataset to investigate the timing of public speaking judgements

Biancardi, B., Chollet, M. and Clavel, C. (2024) Introducing the 3MT_French dataset to investigate the timing of public speaking judgements. Language Resources and Evaluation, (doi: 10.1007/s10579-023-09709-5) (Early Online Publication)

Abstract

In most public speaking datasets, judgements are given after watching the entire performance, or on thin slices randomly selected from the presentations, without focusing on the temporal location of these slices. This does not allow to investigate how people’s judgements develop over time during presentations. This contrasts with primacy and recency theories, which suggest that some moments of the speech could be more salient than others and contribute disproportionately to the perception of the speaker’s performance. To provide novel insights on this phenomenon, we present the 3MT_French dataset. It contains a set of public speaking annotations collected on a crowd-sourcing platform through a novel annotation scheme and protocol. Global evaluation, persuasiveness, perceived self-confidence of the speaker and audience engagement were annotated on different time windows (i.e., the beginning, middle or end of the presentation, or the full video). This new resource will be useful to researchers working on public speaking assessment and training. It will allow to fine-tune the analysis of presentations under a novel perspective relying on socio-cognitive theories rarely studied before in this context, such as first impressions and primacy and recency theories. An exploratory correlation analysis on the annotations provided in the dataset suggests that the early moments of a presentation have a stronger impact on the judgements.

• Item Type: Articles
• Additional Information: This work was partially funded by the Carnot institutes TSN and M.I.N.E.S. under the Inter-Carnot contract 200000830 AI4SoftSkills and the ANR-21-CE33-0016-02 REVITALISE project.
• Keywords: Corpus, public speaking, annotation scheme, first impressions, primacy-recency effect.
• Status: Early Online Publication
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Chollet, Dr Mathieu
• Authors: Biancardi, B., Chollet, M., and Clavel, C.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Language Resources and Evaluation
• Publisher: Springer
• ISSN: 1574-020X
• ISSN (Online): 1574-0218
• Published Online: 23 March 2024
• Copyright Holders: Copyright © The Author(s) 2024
• First Published: First published in Language Resources and Evaluation 2024
• Publisher Policy: Reproduced under a Creative Commons licence