Python Programmers Have GPUs Too: Automatic Python Loop Parallelization with Staged Dependence Analysis

Jacob, D., Trinder, P. and Singer, J. (2019) Python Programmers Have GPUs Too: Automatic Python Loop Parallelization with Staged Dependence Analysis. In: DLS 2019, Athens, Greece, 20-25 Oct 2019, pp. 42-54. ISBN 9781450369961 (doi: 10.1145/3359619.3359743)

193798.pdf - Accepted Version



Python is a popular language for end-user software development in many application domains. End-users want to harness parallel compute resources effectively, by exploiting commodity manycore technology including GPUs. However, existing approaches to parallelism in Python are esoteric, and generally seem too complex for the typical end-user developer. We argue that implicit, or automatic, parallelization is the best way to deliver the benefits of manycore to end-users, since it avoids domain-specific languages, specialist libraries, complex annotations or restrictive language subsets. Auto-parallelization fits the Python philosophy, provides effective performance, and is convenient for non-expert developers. Despite being a dynamic language, we show that Python is a suitable target for auto-parallelization. In an empirical study of 3000+ open-source Python notebooks, we demonstrate that typical loop behaviour ‘in the wild’ is amenable to auto-parallelization. We show that staging the dependence analysis is an effective way to maximize performance. We apply classical dependence analysis techniques, then leverage the Python runtime’s rich introspection capabilities to resolve additional loop bounds and variable types in a just-in-time manner. The parallel loop nest code is then converted to CUDA kernels for GPU execution. We achieve orders of magnitude speedup over baseline interpreted execution and some speedup (up to 50x, although not consistently) over CPU JIT-compiled execution, across 12 loop-intensive standard benchmarks.

Item Type:Conference Proceedings
Glasgow Author(s) Enlighten ID:Singer, Dr Jeremy and Jacob, Dr Dejice and Trinder, Professor Phil
Authors: Jacob, D., Trinder, P., and Singer, J.
College/School:College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright © 2019 Association for Computing Machinery
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record