Enabling object storage via shims for grid middleware

Skipsey, S. C. , De Witt, S., Dewhurst, A., Britton, D. , Roy, G. and Crooks, D. (2015) Enabling object storage via shims for grid middleware. Journal of Physics: Conference Series, 664(4), 042052. (doi: 10.1088/1742-6596/664/4/042052)

Abstract

The Object Store model has quickly become the basis of most commercially successful mass storage infrastructure, backing so-called "Cloud" storage such as Amazon S3, but also underlying the implementation of most parallel distributed storage systems. Many of the assumptions in Object Store design are similar, but not identical, to concepts in the design of Grid Storage Elements, although the requirement for "POSIX-like" filesystem structures on top of SEs makes the disjunction seem larger. As modern Object Stores provide many features that most Grid SEs do not (block level striping, parallel access, automatic file repair, etc.), it is of interest to see how easily we can provide interfaces to typical Object Stores via plugins and shims for Grid tools, and how well experiments can adapt their data models to them. We present evaluation of, and first-deployment experiences with, (for example) Xrootd-Ceph interfaces for direct object-store access, as part of an initiative within GridPP[1] hosted at RAL. Additionally, we discuss the tradeoffs and experience of developing plugins for the currently-popular Ceph parallel distributed filesystem for the GFAL2 access layer, at Glasgow.

• Item Type: Articles (Other)
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Crooks, Dr David and Roy, Dr Gareth and Britton, Professor David and Skipsey, Dr Samuel
• Authors: Skipsey, S. C., De Witt, S., Dewhurst, A., Britton, D., Roy, G., and Crooks, D.
• College/School: College of Science and Engineering > School of Physics and Astronomy
• Journal Name: Journal of Physics: Conference Series
• Publisher: IOP Publishing
• ISSN: 1742-6588
• ISSN (Online): 1742-6596
• Published Online: 23 December 2015
• Copyright Holders: Copyright © 2015 The Authors
• First Published: First published in Journal of Physics: Conference Series 664(4): 042052
• Publisher Policy: Reproduced under a Creative Commons License