Hiding the complexity: building a distributed ATLAS Tier-2 with a single resource interface using ARC middleware

Purdie, S., Stewart, G., Kenyon, M., Skipsey, S. , Washbrook, A., Bhimji, W. and Filipčič, A. (2011) Hiding the complexity: building a distributed ATLAS Tier-2 with a single resource interface using ARC middleware. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP 2010), Taipei, Japan, 18-22 Oct 2010, 072029. (doi: 10.1088/1742-6596/331/7/072029)

Abstract

Since their inception, Grids for high energy physics have found management of data to be the most challenging aspect of operations. This problem has generally been tackled by the experiment's data management framework controlling in fine detail the distribution of data around the grid and the careful brokering of jobs to sites with co-located data. This approach, however, presents experiments with a difficult and complex system to manage as well as introducing a rigidity into the framework which is very far from the original conception of the grid.<p></p> In this paper we describe how the ScotGrid distributed Tier-2, which has sites in Glasgow, Edinburgh and Durham, was presented to ATLAS as a single, unified resource using the ARC middleware stack. In this model the ScotGrid 'data store' is hosted at Glasgow and presented as a single ATLAS storage resource. As jobs are taken from the ATLAS PanDA framework, they are dispatched to the computing cluster with the fastest response time. An ARC compute element at each site then asynchronously stages the data from the data store into a local cache hosted at each site. The job is then launched in the batch system and accesses data locally.<p></p> We discuss the merits of this system compared to other operational models and consider, from the point of view of the resource providers (sites), and from the resource consumers (experiments); and consider issues involved in transitions to this model.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Skipsey, Dr Samuel and Kenyon, Dr Michael and Stewart, Dr Graeme and Purdie, Dr Stuart
• Authors: Purdie, S., Stewart, G., Kenyon, M., Skipsey, S., Washbrook, A., Bhimji, W., and Filipčič, A.
• College/School: College of Science and Engineering > School of Physics and Astronomy
• Journal Name: Journal of Physics: Conference Series
• ISSN: 1742-6596
• Copyright Holders: Copyright © 2010 The Authors
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher

Related URLs

• Organisation