Quantifying the effects of chance in multiple choice and true/false tests: question selection and guessing of answers

Burton, R. F. (2001) Quantifying the effects of chance in multiple choice and true/false tests: question selection and guessing of answers. Assessment and Evaluation in Higher Education, 26(1), pp. 41-50. (doi: 10.1080/02602930020022273)

Abstract

Multiple choice and true/false tests commonly contain only a small proportion of all the questions that could be asked, so that an examinee may be lucky or unlucky in the selection. Random guessing further reduces test reliability when it is not effectively discouraged. Four measures of test unreliability are described that quantify the effects of question selection and guessing, both separately and together - three chosen for immediacy and one for greater mathematical elegance. Their dependence on test length and on the number of answer options per question is quantified. Through one or both random effects, many multiple choice tests are too unreliable for their purpose as currently conducted. Ways of improving test reliability are discussed.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Burton, Dr Richard
• Authors: Burton, R. F.
• College/School: College of Medical Veterinary and Life Sciences > School of Life Sciences
• Journal Name: Assessment and Evaluation in Higher Education
• Publisher: Taylor & Francis
• ISSN: 0260-2938
• ISSN (Online): 1469-297X