Covariance-based weighting for optimal combination of network predictions

Penny, W.D., Husmeier, D. and Roberts, S.J. (1999) Covariance-based weighting for optimal combination of network predictions. In: Proceedings of the 9th International Conference on Artificial Neural Networks. IEEE: Edinburgh, UK, pp. 826-831.

Abstract

This paper introduces a method for calcu- lating the covariance between different neu- ral network solutions. It is based on a generalisation of the delta method for cal- culating the network Hessian and gener- ates what we call the ‘cross-covariance’ ma- trix (its inverse is the ‘cross-Hessian’). Us- ing this matrix we are able to estimate the covariance between network predictions at each point in input space, using train- ing data alone. Whilst this is a signifi- cant result in itself we have also applied the method to the problem of finding optimal linear combinations of models. This results in a ‘covariance-based’ weighted committee, where the weights are input-dependent. If the individual networks are unbiased then the covariance-based weighted committee is optimal in the sense of minimum expected prediction error.

• Item Type: Book Sections
• Status: Published
• Glasgow Author(s) Enlighten ID: Husmeier, Professor Dirk
• Authors: Penny, W.D., Husmeier, D., and Roberts, S.J.
• College/School: College of Science and Engineering > School of Mathematics and Statistics > Statistics
• Publisher: IEEE

Related URLs

• Publisher