Generic exact combinatorial search at HPC scale

MacGregor, R., Archibald, B. and Trinder, P. (2023) Generic exact combinatorial search at HPC scale. International Journal of Parallel Programming, 51(1), pp. 83-106. (doi: 10.1007/s10766-022-00744-3)

[img] Text
276992.pdf - Published Version
Available under License Creative Commons Attribution.

2MB

Abstract

Exact combinatorial search is essential to a wide range of important applications, and there are many large problems that need to be solved quickly. Searches are extremely challenging to parallelise due to a combination of factors, e.g. searches are non-deterministic, dynamic pruning changes the workload, and search tasks have very different runtimes. YewPar is a C++/HPX framework that generalises parallel search by providing a range of sophisticated search skeletons.This paper demonstrates generic high performance combinatorial search, i.e. that a variety of exact combinatorial searches can be easily parallelised for HPC using YewPar. We present a new mechanism for profiling key aspects of YewPar parallel combinatorial search, and demonstrate its value. We exhibit, for the first time, generic exact combinatorial searches at HPC scale. We baseline YewPar against state-of-the-art sequential C++ and C++/OpenMP implementations. We demonstrate that deploying YewPar on an HPC system can dramatically reduce the runtime of large problems, e.g. from days to just 100s. The maximum relative speedups we achieve for an enumeration search are near-linear up to 195(6825) compute-nodes(workers), super-linear for an optimisation search on up to 128(4480) (pruning reduces the workload), and sub-linear for decision searches on up to 64(2240) compute-nodes(workers).

Item Type:Articles
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Macgregor, Mr Ruairidh and Archibald, Dr Blair and Trinder, Professor Phil
Authors: MacGregor, R., Archibald, B., and Trinder, P.
College/School:College of Science and Engineering > School of Computing Science
Journal Name:International Journal of Parallel Programming
Publisher:Springer
ISSN:0885-7458
ISSN (Online):1573-7640
Published Online:07 December 2022
Copyright Holders:Copyright © 2022 The Authors
First Published:First published in International Journal of Parallel Programming 51(1): 83-106
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
173366MaRIONet: Manycore Research Interaction and Opportunities NetworkJeremy SingerEngineering and Physical Sciences Research Council (EPSRC)EP/P006434/1Computing Science
306912Session Types for Reliable Distributed SystemsSimon GayEngineering and Physical Sciences Research Council (EPSRC)EP/T014628/1Computing Science
172422Science of Sensor System Software (SSSS)Muffy CalderEngineering and Physical Sciences Research Council (EPSRC)EP/N007565/1Computing Science