Rating norms should be calculated from cumulative link mixed effects models

Taylor, J. E. , Rousselet, G. A. , Scheepers, C. and Sereno, S. C. (2023) Rating norms should be calculated from cumulative link mixed effects models. Behavior Research Methods, 55, pp. 2175-2196. (doi: 10.3758/s13428-022-01814-7) (PMID:36103049) (PMCID:PMC10439063)

[img] Text
265849.pdf - Published Version
Available under License Creative Commons Attribution.



Studies which provide norms of Likert ratings typically report per-item summary statistics. Traditionally, these summary statistics comprise the mean and the standard deviation (SD) of the ratings, and the number of observations. Such summary statistics can preserve the rank order of items, but provide distorted estimates of the relative distances between items because of the ordinal nature of Likert ratings. Inter-item relations in such ordinal scales can be more appropriately modelled by cumulative link mixed effects models (CLMMs). In a series of simulations, and with a reanalysis of an existing rating norms dataset, we show that CLMMs can be used to more accurately norm items, and can provide summary statistics analogous to the traditionally reported means and SDs, but which are disentangled from participants’ response biases. CLMMs can be applied to solve important statistical issues that exist for more traditional analyses of rating norms.

Item Type:Articles
Additional Information:Funding: This research was supported in part by an Economic and Social Research Council (ESRC) postgraduate fellowship awarded to J. E. Taylor (reference: ES/P000681/l).
Glasgow Author(s) Enlighten ID:Sereno, Dr Sara and Scheepers, Dr Christoph and Rousselet, Dr Guillaume and Taylor, Jack
Authors: Taylor, J. E., Rousselet, G. A., Scheepers, C., and Sereno, S. C.
College/School:College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
Journal Name:Behavior Research Methods
Publisher:Springer for Psychonomic Society
ISSN (Online):1554-3528
Published Online:14 September 2022
Copyright Holders:Copyright © The Author(s) 2022
First Published:First published in Behavior Research Methods 55: 2175–2196
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record