Towards communicative face occlusions: machine detection of hand-over-face gestures

Mahmoud, M. , El-Kaliouby, R. and Goneid, A. (2009) Towards communicative face occlusions: machine detection of hand-over-face gestures. In: Kamel, M. and Campilho, A. (eds.) Image Analysis and Recognition: 6th International Conference, ICIAR 2009, Halifax, Canada, July 6-8, 2009. Proceedings. Series: Lecture notes in computer science (5627). Springer: Berlin, Heidelberg, pp. 481-490. ISBN 9783642026102 (doi: 10.1007/978-3-642-02611-9_48)

Full text not currently available from Enlighten.


Emotional body language constitutes an important channel of non-verbal information. Of this large set, hand-over-face gestures are treated as noise because they occlude facial expressions. In this paper, we propose an alternative facial processing framework where face occlusions instead of being removed, are detected, localized and eventually classified into communicative gestures. We present a video corpus of hand-over-face gestures and describe a multi-stage methodology for detecting and localizing these gestures. For pre-processing, we show that force fields form a better representation of images compared to edge detectors. For feature extraction, detection and localization, we show that Local Binary Patterns outperform Gabor filters in accuracy and speed. Our methodology yields an average detection rate of 97%, is robust to changes in facial expressions, hand shapes, and limited head motion, and preliminary testing with spontaneous videos suggests that it may generalize successfully to naturally evoked videos.

Item Type:Book Sections
Glasgow Author(s) Enlighten ID:Mahmoud, Dr Marwa
Authors: Mahmoud, M., El-Kaliouby, R., and Goneid, A.
College/School:College of Science and Engineering > School of Computing Science

University Staff: Request a correction | Enlighten Editors: Update this record