A Bayesian Reassessment of Nearest-Neighbor Classification

Cucala, L., Marin, J. M., Robert, C. P. and Titterington, D.M. (2009) A Bayesian Reassessment of Nearest-Neighbor Classification. Journal of the American Statistical Association, 104(485), pp. 263-273. (doi: 10.1198/jasa.2009.0125)

Full text not currently available from Enlighten.

Publisher's URL: http://dx.doi.org/10.1198/jasa.2009.0125


The k-nearest-neighbor (knn) procedure is a well-known deterministic method used in supervised classification. This article proposes a reassessment of this approach as a statistical technique derived from a proper probabilistic model; in particular, we modify the assessment found in Holmes and Adams, and evaluated by Manocha and Girolami, where the underlying probabilistic model is not completely well defined. Once provided with a clear probabilistic basis for the knn procedure, we derive computational tools for Bayesian inference on the parameters of the corresponding model. In particular, we assess the difficulties inherent to both pseudo-likelihood and path sampling approximations of an intractable normalizing constant. We implement a correct MCMC sampler based on perfect sampling. When perfect sampling is not available, we use instead a Gibbs sampling approximation. Illustrations of the performance of the corresponding Bayesian classifier are provided for benchmark datasets, demonstrating in particular the limitations of the pseudo-likelihood approximation in this set up

Item Type:Articles
Keywords:Boltzmann Model, Compatible Conditionals, Inference, Likelihood, Markov Chain Monte Carlo Algorithm, Methods, Model, Monte-Carlo Method, Normalizing Constant, Normalizing Constants, Path Sampling, Perfect Sampling, Perfect Simulation, Point-Processes, Pseudo-Likelihood
Glasgow Author(s) Enlighten ID:Titterington, Professor D
Authors: Cucala, L., Marin, J. M., Robert, C. P., and Titterington, D.M.
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Journal Name:Journal of the American Statistical Association

University Staff: Request a correction | Enlighten Editors: Update this record