An Empirical Study of Inter-cluster Resource Orchestration within Federated Cloud Clusters

Lindsay, D., Yeung, G., Elkhatib, Y. and Garraghan, P. (2021) An Empirical Study of Inter-cluster Resource Orchestration within Federated Cloud Clusters. In: 2021 IEEE International Conference on Joint Cloud Computing (JCC), Oxford, UK, 23-26 Aug 2021, ISBN 9781665434805 (doi: 10.1109/JCC53141.2021.00019)

[img] Text
279067.pdf - Accepted Version

807kB

Abstract

Federated clusters are composed of multiple independent clusters of machines interconnected by a resource management system, and possess several advantages over centralized cloud datacenter clusters including seamless provisioning of applications across large geographic regions, greater fault tolerance, and increased cluster resource utilization. However, while existing resource management systems for federated clusters are capable of improving application intra-cluster performance, they do not capture inter-cluster performance in their decision making. This is important given federated clusters must execute a wide variety of applications possessing heterogeneous system architectures, which are a impacted by unique inter-cluster performance conditions such as network latency and localized cluster resource contention. In this work we present an empirical study demonstrating how inter-cluster performance conditions negatively impact federated cluster orchestration systems. We conduct a series of micro-benchmarks under various cluster operational scenarios showing the critical importance in capturing inter-cluster performance for resource orchestration in federated clusters. From this benchmark, we determine precise limitations in existing federated orchestration, and highlight key insights to design future orchestration systems. Findings of notable interest entail different application types exhibiting innate performance affinities across various federated cluster operational conditions, and experience substantial performance degradation from even minor increases to latency (8.7x) and resource contention (12.0x) in comparison to centralized cluster architectures.

Item Type:Conference Proceedings
Additional Information:This work is supported by the UK EPSRC (EP/P031617/1).
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Elkhatib, Dr Yehia
Authors: Lindsay, D., Yeung, G., Elkhatib, Y., and Garraghan, P.
College/School:College of Science and Engineering > School of Computing Science
ISBN:9781665434805
Copyright Holders:Copyright © 2021 IEEE
First Published:First published in 2021 IEEE International Conference on Joint Cloud Computing (JCC)
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record