Evaluating Grouped Spatial Pack Convolutions on Edge CPUs

Gibson, P. , Cano, J. , Turner, J., Crowley, E. J., O’Boyle, M. and Storkey, A. (2020) Evaluating Grouped Spatial Pack Convolutions on Edge CPUs. 16th International Summer School on Advanced Computer Architecture and Compilation for High-Performance and Embedded Systems (ACACES), Online, 06-17 Jul 2020.

Abstract

Grouped convolutions are a drop-in replacement for standard convolutional layers in neural networks. With an adjustable scaling parameter - the number of groups g - they reduce the number of parameters and multiply-accumulate operations (MACs) at the expense of representational power. However, current implementations of grouped convolutions do not perform optimally. In this paper we discuss Grouped Spatial Packed Convolutions (GSPC), a new approach to grouped convolutions, implemented in TVM’s tensor compute language. We analyse a set of networks leveraging the full range of the g parameter, and evaluate their performance in terms of inference time. We observe that GSPC achives the best performance in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4x, 8x and 4x on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/

• Item Type: Conference or Workshop Item
• Status: Published
• Refereed: No
• Glasgow Author(s) Enlighten ID: Cano Reyes, Dr Jose and Gibson, Mr Perry
• Authors: Gibson, P., Cano, J., Turner, J., Crowley, E. J., O’Boyle, M., and Storkey, A.
• College/School: College of Science and Engineering > School of Computing Science
• Copyright Holders: Copyright © 2020 The Authors
• Publisher Policy: Reproduced with the permission of the publisher