Analysing I/O bottlenecks in LHC data analysis on grid storage resources

Bhimji, W., Clark, P., Doidge, M., Hellmich, M. P., Skipsey, S. and Vukotic, I. (2012) Analysing I/O bottlenecks in LHC data analysis on grid storage resources. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP 2012), New York, NY, USA, 21-25 May 2012, 042010. (doi: 10.1088/1742-6596/396/4/042010)

[img]
Preview
Text
95112.pdf - Published Version

1MB

Publisher's URL: http://dx.doi.org/10.1088/1742-6596/396/4/042010

Abstract

We describe recent I/O testing frameworks that we have developed and applied within the UK GridPP Collaboration, the ATLAS experiment and the DPM team, for a variety of distinct purposes. These include benchmarking vendor supplied storage products, discovering scaling limits of SRM solutions, tuning of storage systems for experiment data analysis, evaluating file access protocols, and exploring I/O read patterns of experiment software and their underlying event data models. With multiple grid sites now dealing with petabytes of data, such studies are becoming essential. We describe how the tests build, and improve, on previous work and contrast how the use-cases differ. We also detail the results obtained and the implications for storage hardware, middleware and experiment software.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Skipsey, Dr Samuel
Authors: Bhimji, W., Clark, P., Doidge, M., Hellmich, M. P., Skipsey, S., and Vukotic, I.
College/School:College of Science and Engineering > School of Physics and Astronomy
Journal Name:Journal of Physics: Conference Series
ISSN:1742-6588
Copyright Holders:Copyright © 2012 The Authors
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record