Using immersive virtual reality to recreate the synaesthetic experience

Taylor, R., Savickaite, S., Henderson, S. and Simmons, D. (2023) Using immersive virtual reality to recreate the synaesthetic experience. i-Perception, 14(3), pp. 1-36. (doi: 10.1177/2041669523116630)

Abstract

Synaesthesia is a condition where people experience unusual sensory or cognitive sensations in response to apparently unrelated stimuli. This paper presents two experiments which aimed to examine whether Virtual Reality (VR) technology can be used to recreate the synaesthetic experience. There is a lack of research in this area, with most studies focussing primarily on synaesthetic colors. Experiment 1 aimed to build on previous research by using not only a traditional color-picker but also VR to capture a more nuanced picture of synaesthetic perception. A multiple case study design was used to examine the experiences of six participants in detail. Data gathering took place via Zoom. During the initial data-gathering session, participants used a color-picker to provide grapheme-color associations. After this session, some of the participants’ synaesthetic experiences were recreated using a VR-by-proxy approach. Results indicated that VR is capable of capturing elements of synaesthetic perception that other methods have been unable to, such as texture, small degrees of movement, and 3D structure. Experiment 2 expanded upon these findings by moving beyond the VR-by-proxy approach and asking three participants to recreate their own audiovisual synaesthetic associations in the VR environment. Inductive Thematic Analysis was used to analyze the results of this experiment. The potential of expanding this technique to other forms of perceptual diversity is discussed.

• Item Type: Articles
• Additional Information: Sarune Savickaite is funded by an ESRC-SGSSS collaborative studentship (ES/P000681/1) with industrial partners Sublime Digital Ltd.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Simmons, Dr David and Taylor, Miss Rebecca and Savickaite, Ms Sarune
• Authors: Taylor, R., Savickaite, S., Henderson, S., and Simmons, D.
• College/School: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience; College of Social Sciences > School of Education
• Journal Name: i-Perception
• Publisher: SAGE Publications
• ISSN: 2041-6695
• ISSN (Online): 2041-6695
• Published Online: 04 September 2023
• Copyright Holders: Copyright © The Author(s) 2023
• First Published: First published in i-Perception 14(3):1-36
• Publisher Policy: Reproduced under a Creative Commons license

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
303166		Scottish Graduate School Science Doctoral Training Partnership (DTP)	Mary Beth Kneafsey	Economic and Social Research Council (ESRC)	ES/P000681/1	SS - Academic & Student Administration