Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance

Arapakis, I., Konstas, I. and Jose, J. (2009) Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance. In: Seventeenth ACM International Conference on Multimedia, Beijing, China, 19-24 Oct 2009, pp. 461-470. ISBN 9781605586083 (doi:10.1145/1631272.1631336)

Full text not currently available from Enlighten.

Publisher's URL:


Multimedia search systems face a number of challenges, emanating mainly from the semantic gap problem. Implicit feedback is considered a useful technique in addressing many of the semantic-related issues. By analysing implicit feedback information search systems can tailor the search criteria to address more effectively users' information needs. In this paper we examine whether we could employ affective feedback as an implicit source of evidence, through the aggregation of information from various sensory channels. These channels range between facial expressions to neuro-physiological signals and are regarded as indicative of the user's affective states. The end-goal is to model user affective responses and predict with reasonable accuracy the topical relevance of information items without the help of explicit judgements. For modelling relevance we extract a set of features from the acquired signals and apply different classification techniques, such as Support Vector Machines and K-Nearest Neighbours. The results of our evaluation suggest that the prediction of topical relevance, using the above approach, is feasible and, to a certain extent, implicit feedback models can benefit from incorporating such affective features.

Item Type:Conference Proceedings
Glasgow Author(s) Enlighten ID:Jose, Professor Joemon and Konstas, Mr Ioannis and Arapakis, Mr Ioannis
Authors: Arapakis, I., Konstas, I., and Jose, J.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering > School of Computing Science

University Staff: Request a correction | Enlighten Editors: Update this record