Synthetic Control Methodology as a Tool for Evaluating Public Health Interventions: A Guide to the Method and Case Study

Bouttell, J. , Popham, F. , Lewsey, J. , Robinson, M. and Craig, P. (2017) Synthetic Control Methodology as a Tool for Evaluating Public Health Interventions: A Guide to the Method and Case Study. Lancet Public Health Science Conference, London, UK, 24 Nov 2017.

Abstract

Background: Many public health interventions cannot be evaluated using randomised controlled trials. The alternative methods require a counterfactual of some kind to identify the impact of the intervention. Often, finding an untreated control area is difficult. The synthetic control method builds a counterfactual using a weighted combination of potential control units. We provide a case study with syntax and data available for later use, and summarise the strengths and weaknesses of the method. Methods: We explain the synthetic control method, review its use in public health to date, and describe its implementation through a case study of life expectancy following German reunification. We use publically available data to create a synthetic control, estimate the effect of reunification and conduct a series of placebo tests to check the robustness of the findings. Findings: Our case study showed no effect on life expectancy in West Germany. Advantages of the synthetic control method over the widely-used ‘difference in difference’ method are that it does not rely on parallel pre-implementation trends and allows for time-varying confounders. The counterfactual generated by the synthetic control method is generally a better fit for the pre-implementation trend in the outcome variable than a single or averaged control unit. Limitations include the need for suitable data on both the treated unit and a pool of potential controls, difficulties if the treated unit is an outlier and the inapplicability of traditional statistical tests, although alternatives are available. The method has been under-used in public health to date, with most existing studies taking place in the US. Interpretation: Synthetic control methods are a valuable addition to the range of approaches for evaluating public health interventions when randomisation is impractical. They deserve to be more widely applied, ideally in combination with other methods so that the dependence of findings on particular assumptions can be assessed.

• Item Type: Conference or Workshop Item
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Robinson, Dr Mark and Popham, Dr Frank and Bouttell, Dr Janet and Craig, Professor Peter and Lewsey, Professor Jim
• Authors: Bouttell, J., Popham, F., Lewsey, J., Robinson, M., and Craig, P.
• College/School: College of Medical Veterinary and Life Sciences > School of Health & Wellbeing > Health Economics and Health Technology Assessment; College of Medical Veterinary and Life Sciences > School of Health & Wellbeing > MRC/CSO SPHSU

Related URLs

• Organisation