Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient

Tan, Y., Rérolle, S., Lalitharatne, T. D., van Zalk, N., Jack, R. E. and Nanayakkara, T. (2022) Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient. Scientific Reports, 12, 4200. (doi: 10.1038/s41598-022-08115-1) (PMID:35273296) (PMCID:PMC8913843)

Abstract

Medical training simulators can provide a safe and controlled environment for medical students to practice their physical examination skills. An important source of information for physicians is the visual feedback of involuntary pain facial expressions in response to physical palpation on an affected area of a patient. However, most existing robotic medical training simulators that can capture physical examination behaviours in real-time cannot display facial expressions and comprise a limited range of patient identities in terms of ethnicity and gender. Together, these limitations restrict the utility of medical training simulators because they do not provide medical students with a representative sample of pain facial expressions and face identities, which could result in biased practices. Further, these limitations restrict the utility of such medical simulators to detect and correct early signs of bias in medical training. Here, for the first time, we present a robotic system that can simulate facial expressions of pain in response to palpations, displayed on a range of patient face identities. We use the unique approach of modelling dynamic pain facial expressions using a data-driven perception-based psychophysical method combined with the visuo-haptic inputs of users performing palpations on a robot medical simulator. Specifically, participants performed palpation actions on the abdomen phantom of a simulated patient, which triggered the real-time display of six pain-related facial Action Units (AUs) on a robotic face (MorphFace), each controlled by two pseudo randomly generated transient parameters: rate of change β and activation delay τ. Participants then rated the appropriateness of the facial expression displayed in response to their palpations on a 4-point scale from “strongly disagree” to “strongly agree”. Each participant (n=16, 4 Asian females, 4 Asian males, 4 White females and 4 White males) performed 200 palpation trials on 4 patient identities (Black female, Black male, White female and White male) simulated using MorphFace. Results showed facial expressions rated as most appropriate by all participants comprise a higher rate of change and shorter delay from upper face AUs (around the eyes) to those in the lower face (around the mouth). In contrast, we found that transient parameter values of most appropriate-rated pain facial expressions, palpation forces, and delays between palpation actions varied across participant-simulated patient pairs according to gender and ethnicity. These findings suggest that gender and ethnicity biases affect palpation strategies and the perception of pain facial expressions displayed on MorphFace. We anticipate that our approach will be used to generate physical examination models with diverse patient demographics to reduce erroneous judgments in medical students, and provide focused training to address these errors.

• Item Type: Articles
• Additional Information: This work was supported by the EPSRC Robopatient Grant No EP/T00603X/1 and the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement FACESYNTAX No. 759796).
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Jack, Professor Rachael
• Authors: Tan, Y., Rérolle, S., Lalitharatne, T. D., van Zalk, N., Jack, R. E., and Nanayakkara, T.
• College/School: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Journal Name: Scientific Reports
• Publisher: Nature Research
• ISSN: 2045-2322
• ISSN (Online): 2045-2322
• Copyright Holders: Copyright © 2022 The Authors
• First Published: First published in Scientific Reports 12: 4200
• Publisher Policy: Reproduced under a Creative Commons License

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
304001		Computing the Face Syntax of Social Communication	Rachael Jack	European Research Council (ERC)	759796	Psychology