Understanding face and eye visibility in front-facing cameras of smartphones used in the wild

Khamis, M. , Baier, A., Henze, N., Alt, F. and Bulling, A. (2018) Understanding face and eye visibility in front-facing cameras of smartphones used in the wild. In: 2018 CHI Conference on Human Factors in Computing Systems, Montréal, QC, Canada, 21-26 Apr 2018, p. 280. ISBN 9781450356206 (doi:10.1145/3173574.3173854)

[img]
Preview
Text
170205.pdf - Accepted Version

7MB

Abstract

Commodity mobile devices are now equipped with high-resolution front-facing cameras, allowing applications in biometrics (e.g., FaceID in the iPhone X), facial expression analysis, or gaze interaction. However, it is unknown how often users hold devices in a way that allows capturing their face or eyes, and how this impacts detection accuracy. We collected 25,726 in-the-wild photos, taken from the front-facing camera of smartphones as well as associated application usage logs. We found that the full face is visible about 29% of the time, and that in most cases the face is only partially visible. Furthermore, we identified an influence of users' current activity; for example, when watching videos, the eyes but not the entire face are visible 75% of the time in our dataset. We found that a state-of-the-art face detection algorithm performs poorly against photos taken from front-facing cameras. We discuss how these findings impact mobile applications that leverage face and eye detection, and derive practical implications to address state-of-the art's limitations.

Item Type:Conference Proceedings
Keywords:Eye tracking, face detection, front-facing camera, gaze estimation, in the wild study, mobile device.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Khamis, Dr Mohamed
Authors: Khamis, M., Baier, A., Henze, N., Alt, F., and Bulling, A.
College/School:College of Science and Engineering > School of Computing Science
Publisher:ACM
ISBN:9781450356206
Copyright Holders:Copyright © 2018 The Authors
First Published:First published in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems: 280
Publisher Policy:Reproduced in accordance with the publisher copyright policy

University Staff: Request a correction | Enlighten Editors: Update this record