Combining Embedded Accelerometers with Computer Vision for Recognizing Food Preparation Activities

Stein, S. and McKenna, S. J. (2013) Combining Embedded Accelerometers with Computer Vision for Recognizing Food Preparation Activities. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Zurich, Switzerland, 08-12 Sep 2013, pp. 729-738. ISBN 9781450317702 (doi: 10.1145/2493432.2493482)

Abstract

This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing information captured by these sensor types can considerably improve recognition performance.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Stein, Dr Sebastian
• Authors: Stein, S., and McKenna, S. J.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781450317702