Simpson, K. A. and Pezaros, D. P. (2022) Revisiting the Classics: Online RL in the Programmable Dataplane. In: IEEE/IFIP Network Operations and Management Symposium 2022, Budapest, Hungary, 25-29 April 2022, ISBN 9781665406017 (doi: 10.1109/NOMS54207.2022.9789930)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
262493.pdf - Accepted Version 2MB |
Abstract
Data-driven networking is becoming more capable and widely researched, partly driven by the efficacy of Deep Reinforcement Learning (DRL) algorithms. Yet the complexity of both DRL inference and learning force these tasks to be pushed away from the dataplane to hosts, harming latency-sensitive applications. Online learning of such policies cannot occur in the dataplane, despite being useful techniques when problems evolve or are hard to model.We present OPaL—On Path Learning—the first work to bring online reinforcement learning to the dataplane. OPaL makes online learning possible in constrained SmartNIC hardware by returning to classical RL techniques—avoiding neural networks. Our design allows weak yet highly parallel SmartNIC NPUs to be competitive against commodity x86 hosts, despite having fewer features and slower cores. Compared to hosts, we achieve a 21 × reduction in 99.99th tail inference times to 34 µs, and 9.9 × improvement in online throughput for real-world policy designs. In-NIC execution eliminates PCIe transfers, and our asynchronous compute model ensures minimal impact on traffic carried by a co-hosted P4 dataplane. OPaL’s design scales with additional resources at compile-time to improve upon both decision latency and throughput, and is quickly reconfigurable at runtime compared to reinstalling device firmware.
| Item Type: | Conference Proceedings |
|---|---|
| Additional Information: | This work was supported in part by the Engineering and Physical Sciences Research Council [grants EP/N509668/1, EP/N033957/1]. |
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Simpson, Dr Kyle and Pezaros, Professor Dimitrios |
| Authors: | Simpson, K. A., and Pezaros, D. P. |
| College/School: | College of Science and Engineering > School of Computing Science College of Science and Engineering > School of Engineering |
| ISSN: | 2374-9709 |
| ISBN: | 9781665406017 |
| Published Online: | 09 June 2022 |
| Copyright Holders: | Copyright © 2022 IEEE |
| Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
| Related URLs: |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information