Designing Audio and Tactile Crossmodal Icons for Mobile Devices

Hoggan, E. and Brewster, S.A. (2007) Designing Audio and Tactile Crossmodal Icons for Mobile Devices. ACM Press, pp. 162-169.

Full text not currently available from Enlighten.


This paper reports an experiment into the design of crossmodal icons which can provide an alternative form of output for mobile devices using audio and tactile modalities to communicate information. A complete set of crossmodal icons was created by encoding three dimensions of information in three crossmodal auditory/ tactile parameters. Earcons were used for the audio and Tactons for the tactile crossmodal icons. The experiment investigated absolute identification of audio and tactile crossmodal icons when a user is trained in one modality and tested in the other (and given no training in the other modality) to see if knowledge could be transferred between modalities. We also compared performance when users were static and mobile to see any effects that mobility might have on recognition of the cues. The results showed that if participants were trained in sound with Earcons and then tested with the same messages presented via Tactons they could recognize 85% of messages when stationary and 79% when mobile. When trained with Tactons and tested with Earcons participants could accurately recognize 76.5% of messages when stationary and 78% of messages when mobile. These results suggest that participants can recognize and understand a message in a different modality very effectively. These results will aid designers of mobile displays in creating effective crossmodal cues which require minimal training for users and can provide alternative presentation modalities through which information may be presented if the context requires.

Item Type:Books
Glasgow Author(s) Enlighten ID:Brewster, Professor Stephen and Hoggan, Miss Eve
Authors: Hoggan, E., and Brewster, S.A.
College/School:College of Science and Engineering > School of Computing Science
Publisher:ACM Press

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
353141An investigation of multimodal interaction with tactile displaysStephen BrewsterEngineering & Physical Sciences Research Council (EPSRC)GR/S53244/01Computing Science