Q-learning: flexible learning about useful utilities

Moodie, E. E.M., Dean, N. and Sun, Y. R. (2014) Q-learning: flexible learning about useful utilities. Statistics in Biosciences, 6(2), pp. 223-243. (doi:10.1007/s12561-013-9103-z)

[img] Text
85869.pdf - Accepted Version

531kB

Publisher's URL: http://dx.doi.org/10.1007/s12561-013-9103-z

Abstract

Dynamic treatment regimes are fast becoming an important part of medicine, with the corresponding change in emphasis from treatment of the disease to treatment of the individual patient. Because of the limited number of trials to evaluate personally tailored treatment sequences, inferring optimal treatment regimes from observational data has increased importance. Q-learning is a popular method for estimating the optimal treatment regime, originally in randomized trials but more recently also in observational data. Previous applications of Q-learning have largely been restricted to continuous utility end-points with linear relationships. This paper is the first attempt at both extending the framework to discrete utilities and implementing the modelling of covariates from linear to more flexible modelling using the generalized additive model (GAM) framework. Simulated data results show that the GAM adapted Q-learning typically outperforms Q-learning with linear models and other frequently-used methods based on propensity scores in terms of coverage and bias/MSE. This represents a promising step toward a more fully general Q-learning approach to estimating optimal dynamic treatment regimes.

Item Type:Articles
Keywords:Dynamic Treatment Regimes, Q-learning, Generalized Additive Models, Discrete data
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Dean, Dr Nema
Authors: Moodie, E. E.M., Dean, N., and Sun, Y. R.
Subjects:H Social Sciences > HA Statistics
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Research Group:Statistical Methodology
Journal Name:Statistics in Biosciences
Journal Abbr.:Stat Biosci
Publisher:Springer US
ISSN:1867-1764
ISSN (Online):1867-1772
Copyright Holders:Copyright © 2014 Springer
First Published:First published in Statistics in Biosciences 6()2):223-243
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher.

University Staff: Request a correction | Enlighten Editors: Update this record