Albers, P. N. et al. (2023) Natural experiments for the evaluation of place-based public health interventions: a methodology scoping review. Frontiers in Public Health, 11, 1192055. (doi: 10.3389/fpubh.2023.1192055) (PMID:37427271) (PMCID:PMC10323422)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
302421.pdf - Published Version Available under License Creative Commons Attribution. 912kB |
Abstract
Introduction: Place-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the as-if randomization assumption. Methods: A systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of as-if randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed ‘as-if ' randomization for each. Results: 366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable as-if randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of as-if randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences. Conclusion: NEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of as-if randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs.
| Item Type: | Articles |
|---|---|
| Additional Information: | This study was funded by the National Institute for Health Research (NIHR) School for Public Health Research (Grant Reference Number PD-SPH-2015). The funder had no input in the writing of the manuscript or decision to submit for publication. The NIHR School for Public Health Research is a partnership between the Universities of Sheffield, Bristol, Cambridge, Imperial, and University College London, The London School for Hygiene and Tropical Medicine (LSHTM), LiLaC—a collaboration between the Universities of Liverpool and Lancaster; and Fuse—The Center for Translational Research in Public Health a collaboration between Newcastle, Durham, Northumbria, Sunderland and Teesside Universities. FV and RJ are partly funded by National Institute for Health Research Applied Research Collaboration West (NIHR ARC West) at University Hospitals Bristol NHS Foundation Trust. KT works in the MRC Integrative Epidemiology Unit, which was supported by the Medical Research Council (MRC) and the University of Bristol [MC_UU_00011/3]. DT-R was funded by the Medical Research Council (MRC) on a Clinician Scientist Fellowship (MR/P008577/1). CR was funded by an NIHR Doctoral Fellowship (NIHR301784). PC, MG, and MC acknowledge funding from the Medical Research Council (MC_UU_00022/2) and Scottish Government Chief Scientist Office (SPHSU17). |
| Keywords: | Place-based, evaluations, public health policy, quasi experiments, public health, natural experiments. |
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Craig, Professor Peter and Gibson, Dr Marcia and Campbell, Ms Mhairi |
| Authors: | Albers, P. N., Rinaldi, C., Brown, H., Mason, K. E., d'Apice, K., McGill, E., McQuire, C., Craig, P., Laverty, A. A., Beeson, M., Campbell, M., Egan, M., Gibson, M., Fuller, M., Dillon, A., Taylor-Robinson, D., Jago, R., Tilling, K., Barr, B., Sniehotta, F. F., Hickman, M., Millett, C. J., and de Vocht, F. |
| College/School: | College of Medical Veterinary and Life Sciences > School of Health & Wellbeing > MRC/CSO SPHSU |
| Journal Name: | Frontiers in Public Health |
| Publisher: | Frontiers Media |
| ISSN: | 2296-2565 |
| ISSN (Online): | 2296-2565 |
| Copyright Holders: | Copyright © 2023 Albers, Rinaldi, Brown, Mason, d'Apice, McGill, McQuire, Craig, Laverty, Beeson, Campbell, Egan, Gibson, Fuller, Dillon, Taylor-Robinson, Jago, Tilling, Barr, Sniehotta, Hickman, Millett and de Vocht |
| First Published: | First published in Frontiers in Public Health 11: 1192055 |
| Publisher Policy: | Reproduced under a Creative Commons License |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information