Online RL in the Programmable Dataplane with OPaL

Simpson, K. A. and Pezaros, D. (2021) Online RL in the Programmable Dataplane with OPaL. In: 17th International Conference on emerging Networking EXperiments and Technologies (CONEXT 2021 Posters), Munich, Germany, 7-10 Dec 2021, pp. 471-472. ISBN 9781450390989 (doi: 10.1145/3485983.3493345)

[img] Text
258124.pdf - Accepted Version

2MB

Abstract

Reinforcement learning (RL) is a key tool in data-driven networking for learning to control systems online. While recent research has shown how to offload machine learning tasks to the dataplane (reducing processing latency), online learning remains an open challenge unless the model is moved back to a host CPU, harming latency-sensitive applications. Our poster introduces OPaL---On Path Learning---the first work to bring online reinforcement learning to the dataplane. OPaL makes online learning possible in SmartNIC/NPU hardware by returning to classical RL techniques---avoiding neural networks. This simplifies update logic, enabling online learning, and benefits well from the parallelism common to SmartNICs. We show that our implementation on Netronome SmartNIC hardware offers concrete latency improvements over host execution.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Simpson, Mr Kyle and Pezaros, Professor Dimitrios
Authors: Simpson, K. A., and Pezaros, D.
College/School:College of Science and Engineering > School of Computing Science
ISBN:9781450390989
Published Online:03 December 2021
Copyright Holders:Copyright © 2021 Copyright held by the owner/author(s)
First Published:First published in CoNEXT '21: Proceedings of the 17th International Conference on emerging Networking EXperiments and Technologies
Publisher Policy:Reproduced in accordance with the publisher copyright policy
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record