Bhimji, W., Clark, P., Doidge, M., Hellmich, M. P., Skipsey, S. and Vukotic, I. (2012) Analysing I/O bottlenecks in LHC data analysis on grid storage resources. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP 2012), New York, NY, USA, 21-25 May 2012, 042010. (doi: 10.1088/1742-6596/396/4/042010)
|
Text
95112.pdf - Published Version 1MB |
Publisher's URL: http://dx.doi.org/10.1088/1742-6596/396/4/042010
Abstract
We describe recent I/O testing frameworks that we have developed and applied within the UK GridPP Collaboration, the ATLAS experiment and the DPM team, for a variety of distinct purposes. These include benchmarking vendor supplied storage products, discovering scaling limits of SRM solutions, tuning of storage systems for experiment data analysis, evaluating file access protocols, and exploring I/O read patterns of experiment software and their underlying event data models. With multiple grid sites now dealing with petabytes of data, such studies are becoming essential. We describe how the tests build, and improve, on previous work and contrast how the use-cases differ. We also detail the results obtained and the implications for storage hardware, middleware and experiment software.
Item Type: | Conference Proceedings |
---|---|
Status: | Published |
Refereed: | Yes |
Glasgow Author(s) Enlighten ID: | Skipsey, Dr Samuel |
Authors: | Bhimji, W., Clark, P., Doidge, M., Hellmich, M. P., Skipsey, S., and Vukotic, I. |
College/School: | College of Science and Engineering > School of Physics and Astronomy |
Journal Name: | Journal of Physics: Conference Series |
ISSN: | 1742-6588 |
Copyright Holders: | Copyright © 2012 The Authors |
Publisher Policy: | Reproduced in accordance with the copyright policy of the publisher |
University Staff: Request a correction | Enlighten Editors: Update this record