Rating norms should be calculated from cumulative link mixed effects models

Taylor, J. E. , Rousselet, G. A. , Scheepers, C. and Sereno, S. C. (2022) Rating norms should be calculated from cumulative link mixed effects models. Behavior Research Methods, (doi: 10.3758/s13428-022-01814-7) (PMID:36103049) (Early Online Publication)

Abstract

Studies which provide norms of Likert ratings typically report per-item summary statistics. Traditionally, these summary statistics comprise the mean and the standard deviation (SD) of the ratings, and the number of observations. Such summary statistics can preserve the rank order of items, but provide distorted estimates of the relative distances between items because of the ordinal nature of Likert ratings. Inter-item relations in such ordinal scales can be more appropriately modelled by cumulative link mixed effects models (CLMMs). In a series of simulations, and with a reanalysis of an existing rating norms dataset, we show that CLMMs can be used to more accurately norm items, and can provide summary statistics analogous to the traditionally reported means and SDs, but which are disentangled from participants’ response biases. CLMMs can be applied to solve important statistical issues that exist for more traditional analyses of rating norms.

• Item Type: Articles
• Additional Information: Funding: This research was supported in part by an Economic and Social Research Council (ESRC) postgraduate fellowship awarded to J. E. Taylor (reference: ES/P000681/l).
• Status: Early Online Publication
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Scheepers, Dr Christoph and Rousselet, Dr Guillaume and Taylor, Jack and Sereno, Dr Sara
• Authors: Taylor, J. E., Rousselet, G. A., Scheepers, C., and Sereno, S. C.
• College/School: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Journal Name: Behavior Research Methods
• Publisher: Springer
• ISSN: 1554-351X
• ISSN (Online): 1554-3528
• Published Online: 14 September 2022
• Copyright Holders: Copyright © The Author(s) 2022
• First Published: First published in Behavior Research Methods 2022
• Publisher Policy: Reproduced under a Creative Commons License