Closed-loop deep learning: generating forward models with backpropagation

Daryanavard, S. and Porr, B. (2020) Closed-loop deep learning: generating forward models with backpropagation. Neural Computation, 32(11), pp. 2122-2144. (doi: 10.1162/neco_a_01317) (PMID:32946708)

[img] Text
226317.pdf - Published Version



A reflex is a simple closed-loop control approach that tries to minimize an error but fails to do so because it will always react too late. An adaptive algorithm can use this error to learn a forward model with the help of predictive cues. For example, a driver learns to improve steering by looking ahead to avoid steering in the last minute. In order to process complex cues such as the road ahead, deep learning is a natural choice. However, this is usually achieved only indirectly by employing deep reinforcement learning having a discrete state space. Here, we show how this can be directly achieved by embedding deep learning into a closed-loop system and preserving its continuous processing. We show in z-space specifically how error backpropagation can be achieved and in general how gradient-based approaches can be analyzed in such closed-loop scenarios. The performance of this learning paradigm is demonstrated using a line follower in simulation and on a real robot that shows very fast and continuous learning.

Item Type:Articles
Glasgow Author(s) Enlighten ID:Porr, Dr Bernd and Daryanavard, Sama
Authors: Daryanavard, S., and Porr, B.
College/School:College of Science and Engineering > School of Engineering > Biomedical Engineering
Journal Name:Neural Computation
Publisher:MIT Press
ISSN (Online):1530-888X
Published Online:20 October 2020
Copyright Holders:Copyright © 2020 Massachusetts Institute of Technology
First Published:First published in Neural Computation 32(11): 2122-2144
Publisher Policy:Reproduced in accordance with the publisher copyright policy

University Staff: Request a correction | Enlighten Editors: Update this record