Accelerating CNN Inference on Long Vector Architectures Via Co-design

Rani Gupta, S., Papadopoulou, N. and Pericàs, M. (2023) Accelerating CNN Inference on Long Vector Architectures Via Co-design. In: 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS), St. Petersburg, FL, USA, 15-19 May 2023, pp. 145-155. ISBN 9798350337662 (doi: 10.1109/ipdps54959.2023.00024)

Abstract

CPU-based inference can be deployed as an alternative to off-chip accelerators. In this context, emerging vector architectures are a promising option, owing to their high efficiency. Yet the large design space of convolutional algorithms and hardware implementations makes the selection of design options challenging. In this paper, we present our ongoing research into co-designing future vector architectures for CPU-based Convolutional Neural Networks (CNN) inference focusing on the im2col+GEMM and Winograd kernels. Using the Gem5 simulator we explore the impact of several hardware microarchitectural features including (i) vector lanes, (ii) vector lengths, (iii) cache sizes, and (iv) options for integrating the vector unit into the CPU pipeline. In the context of im2col+GEMM, we study the impact of several BLIS-like algorithmic optimizations such as (1) utilization of vector registers, (2) loop unrolling, (3) loop reorder, (4) manual vectorization, (5) prefetching, and (6) packing of matrices, on the RISC-V Vector Extension and ARM-SVE ISAs. We use the YOLOv3 and VGG16 network models for our evaluation. Our co-design study shows that BLIS-like optimizations are not beneficial to all types of vector microarchitectures. We additionally demonstrate that longer vector lengths (of at least 8192 bits) and larger caches (of 256MB) can boost performance by 5×, with our optimized CNN kernels, compared to a vector length of 512-bit and 1MB of L2 cache. In the context of Winograd, we present our novel approach of inter-tile parallelization across the input/output channels by using 8×8 tiles per channel to vectorize the algorithm on vector length agnostic (VLA) architectures. Our method exploits longer vector lengths and offers high memory reuse, resulting in performance improvement of up to 2.4× for non-strided convolutional layers with 3×3 kernel size, compared to our optimized im2col+GEMM approach on the Fujitsu A64FX processor. Our co-design study furthermore reveals that Winograd requires smaller cache sizes (up to 64MB) compared to im2col+GEMM.

• Item Type: Conference Proceedings
• Additional Information: This work has been supported by the Swedish Research Council via registration number 2020-04892. The simulations were enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at NSC partially funded by the Swedish Research Council through grant agreement no. 2018-05973.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Papadopoulou, Dr Nikela
• Authors: Rani Gupta, S., Papadopoulou, N., and Pericàs, M.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 1530-2075
• ISBN: 9798350337662
• Copyright Holders: Copyright © 2023 The Authors
• First Published: First published in Proceedings of the 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher