Statistical challenges of high-dimensional data

Johnstone, I.M. and Titterington, D.M. (2009) Statistical challenges of high-dimensional data. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 367(1906), pp. 4237-4253. (doi:10.1098/rsta.2009.0159)

Full text not currently available from Enlighten.

Publisher's URL: http://dx.doi.org/10.1098/rsta.2009.0159

Abstract

Modern applications of statistical theory and methods can involve extremely large datasets, often with huge numbers of measurements on each of a comparatively small number of experimental units. New methodology and accompanying theory have emerged in response: the goal of this Theme Issue is to illustrate a number of these recent developments. This overview article introduces the difficulties that arise with high-dimensional data in the context of the very familiar linear statistical model: we give a taste of what can nevertheless be achieved when the parameter vector of interest is sparse, that is, contains many zero elements. We describe other ways of identifying low-dimensional subspaces of the data space that contain all useful information. The topic of classification is then reviewed along with the problem of identifying, from within a very large set, the variables that help to classify observations. Brief mention is made of the visualization of high-dimensional data and ways to handle computational problems in Bayesian analysis are described. At appropriate points, reference is made to the other papers in the issue

Item Type:Articles
Keywords:Bayesian analysis Classification cluster analysis DANTZIG SELECTOR EIGENMAPS High-dimensional data LASSO Methods MODEL RARE REDUCTION REGRESSION Sparsity USEFUL FEATURES WEAK
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Titterington, Professor Michael
Authors: Johnstone, I.M., and Titterington, D.M.
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Journal Name:Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Publisher:Royal Society
ISSN:1364-503X

University Staff: Request a correction | Enlighten Editors: Update this record