High-speed object detection with a single-photon time-of-flight image sensor

Mora-Martín, G., Turpin, A. , Ruget, A., Halimi, A., Henderson, R., Leach, J. and Gyongy, I. (2021) High-speed object detection with a single-photon time-of-flight image sensor. Optics Express, 29(21), pp. 33184-33196. (doi: 10.1364/OE.435619) (PMID:34809135)

[img] Text
255597.pdf - Published Version
Available under License Creative Commons Attribution.

4MB

Abstract

3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and autonomous systems. Single-photon avalanche diodes (SPADs) are one of the enabling technologies providing accurate depth data even over long ranges. By developing SPADs in array format with integrated processing combined with pulsed, flood-type illumination, high-speed 3D capture is possible. However, array sizes tend to be relatively small, limiting the lateral resolution of the resulting depth maps and, consequently, the information that can be extracted from the image for applications such as object detection. In this paper, we demonstrate that these limitations can be overcome through the use of convolutional neural networks (CNNs) for high-performance object detection. We present outdoor results from a portable SPAD camera system that outputs 16-bin photon timing histograms with 64×32 spatial resolution, with each histogram containing thousands of photons. The results, obtained with exposure times down to 2 ms (equivalent to 500 FPS) and in signal-to-background (SBR) ratios as low as 0.05, point to the advantages of providing the CNN with full histogram data rather than point clouds alone. Alternatively, a combination of point cloud and active intensity data may be used as input, for a similar level of performance. In either case, the GPU-accelerated processing time is less than 1 ms per frame, leading to an overall latency (image acquisition plus processing) in the millisecond range, making the results relevant for safety-critical computer vision applications which would benefit from faster than human reaction times.

Item Type:Articles
Additional Information:Funding: Defence Science and Technology Laboratory (DSTLX1000147844); Royal Academy of Engineering (RF/201718/17128); Engineering and Physical Sciences Research Council (EP/M01326X/1, EP/S001638/1).
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Turpin, Dr Alejandro
Authors: Mora-Martín, G., Turpin, A., Ruget, A., Halimi, A., Henderson, R., Leach, J., and Gyongy, I.
College/School:College of Science and Engineering > School of Computing Science
College of Science and Engineering > School of Physics and Astronomy
Journal Name:Optics Express
Publisher:Optical Society of America
ISSN:1094-4087
ISSN (Online):1094-4087
Published Online:29 September 2021
Copyright Holders:Copyright © 2021 The Authors
First Published:First published in Optics Express 29(21): 33184-33196
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
190841UK Quantum Technology Hub in Enhanced Quantum ImagingMiles PadgettEngineering and Physical Sciences Research Council (EPSRC)EP/M01326X/1P&S - Physics & Astronomy