On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance

Xue, J. H. and Titterington, D. M. (2010) On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance. Computational Statistics and Data Analysis, 54(2), pp. 438-451. (doi: 10.1016/j.csda.2009.09.011)

Full text not currently available from Enlighten.

Publisher's URL: http://dx.doi.org/10.1016/j.csda.2009.09.011


The interpretation of generative, discriminative and hybrid approaches to classification is discussed, in particular for the generative-discriminative tradeoff (GDT), a hybrid approach. The asymptotic efficiency of the GDT relative to that of its generative or discriminative counterpart, is presented theoretically and, by using linear normal discrimination as an example, numerically. On real and simulated datasets, the classification performance of the GDT is compared with those of normal-based linear discriminant analysis (LDA) and linear logistic regression (LLR). Four arguments are made as follows. First, the GDT is a generative model integrating both discriminative and generative learning. It is therefore subject to model misspecification of the data-generating process and hindered by complex optimisation. Secondly, among the three approaches being compared, the asymptotic efficiency of the GDT is higher than that of the discriminative approach but lower than that of the generative approach, when no model misspecification occurs. Thirdly, without model misspecification, LDA performs the best; with model misspecification, LLR or the GDT with an optimal, large weight on its discriminative component may perform the best. Finally, LLR is affected by the imbalance between groups of data. (C) 2009 Elsevier B.V. All rights reserved

Item Type:Articles
Keywords:Classification Data-generating process DISCRIMINANT-ANALYSIS DISCRIMINATION ERROR RATE Linear discriminant analysis Linear logistic regression LOGISTIC-REGRESSION MODEL REGRESSION
Glasgow Author(s) Enlighten ID:Titterington, Professor D
Authors: Xue, J. H., and Titterington, D. M.
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Journal Name:Computational Statistics and Data Analysis

University Staff: Request a correction | Enlighten Editors: Update this record