Deep learning for real-time single-pixel video

Higham, C. F. , Murray-Smith, R. , Padgett, M. J. and Edgar, M. P. (2018) Deep learning for real-time single-pixel video. Scientific Reports, 8, 2369. (doi: 10.1038/s41598-018-20521-y) (PMID:29403059) (PMCID:PMC5799195)

155592.pdf - Published Version
Available under License Creative Commons Attribution.



Single-pixel cameras capture images without the requirement for a multi-pixel sensor, enabling the use of state-of-the-art detector technologies and providing a potentially low-cost solution for sensing beyond the visible spectrum. One limitation of single-pixel cameras is the inherent trade-off between image resolution and frame rate, with current compressive (compressed) sensing techniques being unable to support real-time video. In this work we demonstrate the application of deep learning with convolutional auto-encoder networks to recover real-time 128 × 128 pixel video at 30 frames-per-second from a single-pixel camera sampling at a compression ratio of 2%. In addition, by training the network on a large database of images we are able to optimise the first layer of the convolutional network, equivalent to optimising the basis used for scanning the image intensities. This work develops and implements a novel approach to solving the inverse problem for single-pixel cameras efficiently and represents a significant step towards real-time operation of computational imagers. By learning from examples in a particular context, our approach opens up the possibility of high resolution for task-specific adaptation, with importance for applications in gas sensing, 3D imaging and metrology.

Item Type:Articles
Glasgow Author(s) Enlighten ID:Murray-Smith, Professor Roderick and Higham, Dr Catherine and Padgett, Professor Miles and Edgar, Dr Matthew
Authors: Higham, C. F., Murray-Smith, R., Padgett, M. J., and Edgar, M. P.
College/School:College of Science and Engineering > School of Computing Science
College of Science and Engineering > School of Physics and Astronomy
Journal Name:Scientific Reports
Publisher:Nature Research
ISSN (Online):2045-2322
Copyright Holders:Copyright © 2018 The Authors
First Published:First published in Scientific Reports 8: 2369
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
6672319UK Quantum Technology Hub in Enhanced Quantum ImagingMiles PadgettEngineering and Physical Sciences Research Council (EPSRC)EP/M01326X/1S&E P&A - PHYSICS & ASTRONOMY
624501TWISTS - Twists & more: the complex shape of lightMiles PadgettEuropean Research Council (ERC)340507S&E P&A - PHYSICS & ASTRONOMY