Covariance-based weighting for optimal combination of network predictions

Penny, W.D., Husmeier, D. and Roberts, S.J. (1999) Covariance-based weighting for optimal combination of network predictions. In: Proceedings of the 9th International Conference on Artificial Neural Networks. IEEE: Edinburgh, UK, pp. 826-831.

Full text not currently available from Enlighten.

Publisher's URL:


This paper introduces a method for calcu- lating the covariance between different neu- ral network solutions. It is based on a generalisation of the delta method for cal- culating the network Hessian and gener- ates what we call the ‘cross-covariance’ ma- trix (its inverse is the ‘cross-Hessian’). Us- ing this matrix we are able to estimate the covariance between network predictions at each point in input space, using train- ing data alone. Whilst this is a signifi- cant result in itself we have also applied the method to the problem of finding optimal linear combinations of models. This results in a ‘covariance-based’ weighted committee, where the weights are input-dependent. If the individual networks are unbiased then the covariance-based weighted committee is optimal in the sense of minimum expected prediction error.

Item Type:Book Sections
Glasgow Author(s) Enlighten ID:Husmeier, Professor Dirk
Authors: Penny, W.D., Husmeier, D., and Roberts, S.J.
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record