Sign Spotting Using Hierarchical Sequential Patterns With Temporal Intervals

Ong, E.-J., Koller, O., Pugeault, N. and Bowden, R. (2014) Sign Spotting Using Hierarchical Sequential Patterns With Temporal Intervals. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OS, USA, 23-28 Jun 2014, pp. 1931-1938. ISBN 9781479951185 (doi: 10.1109/CVPR.2014.248)

Full text not currently available from Enlighten.

Abstract

This paper tackles the problem of spotting a set of signs occuring in videos with sequences of signs. To achieve this, we propose to model the spatio-temporal signatures of a sign using an extension of sequential patterns that contain temporal intervals called Sequential Interval Patterns (SIP). We then propose a novel multi-class classifier that organises different sequential interval patterns in a hierarchical tree structure called a Hierarchical SIP Tree (HSP-Tree). This allows one to exploit any subsequence sharing that exists between different SIPs of different classes. Multiple trees are then combined together into a forest of HSP-Trees resulting in a strong classifier that can be used to spot signs. We then show how the HSP-Forest can be used to spot sequences of signs that occur in an input video. We have evaluated the method on both concatenated sequences of isolated signs and continuous sign sequences. We also show that the proposed method is superior in robustness and accuracy to a state of the art sign recogniser when applied to spotting a sequence of signs.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Pugeault, Dr Nicolas
Authors: Ong, E.-J., Koller, O., Pugeault, N., and Bowden, R.
College/School:College of Science and Engineering > School of Computing Science
ISSN:1063-6919
ISBN:9781479951185
Published Online:25 September 2014

University Staff: Request a correction | Enlighten Editors: Update this record