Closed-loop deep learning: generating forward models with backpropagation

Daryanavard, S. and Porr, B. (2020) Closed-loop deep learning: generating forward models with backpropagation. Neural Computation, 32(11), pp. 2122-2144. (doi: 10.1162/neco_a_01317) (PMID:32946708)

Abstract

A reflex is a simple closed-loop control approach that tries to minimize an error but fails to do so because it will always react too late. An adaptive algorithm can use this error to learn a forward model with the help of predictive cues. For example, a driver learns to improve steering by looking ahead to avoid steering in the last minute. In order to process complex cues such as the road ahead, deep learning is a natural choice. However, this is usually achieved only indirectly by employing deep reinforcement learning having a discrete state space. Here, we show how this can be directly achieved by embedding deep learning into a closed-loop system and preserving its continuous processing. We show in z-space specifically how error backpropagation can be achieved and in general how gradient-based approaches can be analyzed in such closed-loop scenarios. The performance of this learning paradigm is demonstrated using a line follower in simulation and on a real robot that shows very fast and continuous learning.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Porr, Dr Bernd and Daryanavard, Sama
• Authors: Daryanavard, S., and Porr, B.
• College/School: College of Science and Engineering > School of Engineering > Biomedical Engineering
• Journal Name: Neural Computation
• Publisher: MIT Press
• ISSN: 0899-7667
• ISSN (Online): 1530-888X
• Published Online: 20 October 2020
• Copyright Holders: Copyright © 2020 Massachusetts Institute of Technology
• First Published: First published in Neural Computation 32(11): 2122-2144
• Publisher Policy: Reproduced in accordance with the publisher copyright policy