Scalable Persistent Storage for Erlang

Ghaffari, A., Chechina, N., Trinder, P. and Meredith, J. (2013) Scalable Persistent Storage for Erlang. In: Twelfth ACM SIGPLAN Workshop on Erlang, Boston, MA, USA, 25-27 Sep 2013, pp. 73-74. ISBN 9781450323857 (doi:10.1145/2505305.2505315)

[img]
Preview
Text
107445.pdf - Accepted Version

639kB

Abstract

The many core revolution makes scalability a key property. The RELEASE project aims to improve the scalability of Erlang on emergent commodity architectures with 100,000 cores. Such architectures require scalable and available persistent storage on up to 100 hosts. We enumerate the requirements for scalable and available persistent storage, and evaluate four popular Erlang DBMSs against these requirements. This analysis shows that Mnesia and CouchDB are not suitable persistent storage at our target scale, but Dynamo-like NoSQL DataBase Management Systems (DBMSs) such as Cassandra and Riak potentially are. We investigate the current scalability limits of the Riak 1.1.1 NoSQL DBMS in practice on a 100-node cluster. We establish for the first time scientifically the scalability limit of Riak as 60 nodes on the Kalkyl cluster, thereby confirming developer folklore. We show that resources like memory, disk, and network do not limit the scalability of Riak. By instrumenting Erlang/OTP and Riak libraries we identify a specific Riak functionality that limits scalability. We outline how later releases of Riak are refactored to eliminate the scalability bottlenecks. We conclude that Dynamo-style NoSQL DBMSs provide scalable and available persistent storage for Erlang in general, and for our RELEASE target architecture in particular.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Trinder, Professor Phil and Chechina, Dr Natalia and Ghaffari, Mr Amir
Authors: Ghaffari, A., Chechina, N., Trinder, P., and Meredith, J.
College/School:College of Science and Engineering > School of Computing Science
ISBN:9781450323857
Copyright Holders:Copyright © 2013 ACM
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
635432RELEASEPhil TrinderEuropean Commission (EC)UNSPECIFIEDCOM - COMPUTING SCIENCE