Online RL in the Programmable Dataplane with OPaL

Simpson, K. A. and Pezaros, D. (2021) Online RL in the Programmable Dataplane with OPaL. In: 17th International Conference on emerging Networking EXperiments and Technologies (CONEXT 2021 Posters), Munich, Germany, 7-10 Dec 2021, pp. 471-472. ISBN 9781450390989 (doi: 10.1145/3485983.3493345)

Abstract

Reinforcement learning (RL) is a key tool in data-driven networking for learning to control systems online. While recent research has shown how to offload machine learning tasks to the dataplane (reducing processing latency), online learning remains an open challenge unless the model is moved back to a host CPU, harming latency-sensitive applications. Our poster introduces OPaL---On Path Learning---the first work to bring online reinforcement learning to the dataplane. OPaL makes online learning possible in SmartNIC/NPU hardware by returning to classical RL techniques---avoiding neural networks. This simplifies update logic, enabling online learning, and benefits well from the parallelism common to SmartNICs. We show that our implementation on Netronome SmartNIC hardware offers concrete latency improvements over host execution.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Simpson, Dr Kyle and Pezaros, Professor Dimitrios
• Authors: Simpson, K. A., and Pezaros, D.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781450390989
• Published Online: 03 December 2021
• Copyright Holders: Copyright © 2021 Copyright held by the owner/author(s)
• First Published: First published in CoNEXT '21: Proceedings of the 17th International Conference on emerging Networking EXperiments and Technologies
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation
• Publisher