CoCoPeLia: Communication-Computation Overlap Prediction for Efficient Linear Algebra on GPUs

Anastasiadis, P., Papadopoulou, N. , Goumas, G. and Koziris, N. (2021) CoCoPeLia: Communication-Computation Overlap Prediction for Efficient Linear Algebra on GPUs. In: 2021 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Stony Brook, NY, USA, 28-30 Mar 2021, pp. 36-47. ISBN 9781728186436 (doi: 10.1109/ispass51385.2021.00015)

Abstract

Graphics Processing Units (GPUs) are well established in HPC systems and frequently used to accelerate linear algebra routines. Since data transfers pose a severe bottleneck for GPU offloading, modern GPUs provide the ability to overlap communication with computation by splitting the problem to fine-grained sub-kernels that are executed in a pipelined manner. This optimization is currently underutilized by GPU BLAS libraries, since it requires an approach to select an efficient tiling size, which in turn leads to a challenging problem that needs to consider routine, system, data, and problem-specific characteristics. In this work, we introduce an elaborate 3-way concurrency model for GPU BLAS offload time that considers previously neglected features regarding data access and machine behavior. We then incorporate our model in an automated, end-to-end framework (called CoCoPeLia) that supports overlap prediction, tile selection and effective tile scheduling. We validate our model's efficacy for dgemm, sgemm, and daxpy on two testbeds, with our experimental results showing that it achieves significantly lower prediction error than previous models and provides near-optimal tiling sizes for all problems. We also demonstrate that CoCoPeLia leads to considerable performance improvements compared to the state of the art BLAS routine implementations for GPUs.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Papadopoulou, Dr Nikela
• Authors: Anastasiadis, P., Papadopoulou, N., Goumas, G., and Koziris, N.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: 2021 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
• ISBN: 9781728186436
• Copyright Holders: Copyright © 2021 IEEE
• First Published: First published in Proceedings of the 2021 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher