Understanding Public Evaluation: Quantifying Experimenter Intervention

Williamson, J. R. and Williamson, J. (2017) Understanding Public Evaluation: Quantifying Experimenter Intervention. In: Proceedings of ACM SIGCHI 2017, Denver, CO, USA, 6-11 May 2017, pp. 3414-3425. ISBN 9781450346559 (doi: 10.1145/3025453.3025598)

Abstract

Public evaluations are popular because some research questions can only be answered by turning “to the wild.” Different approaches place experimenters in different roles during deployment, which has implications for the kinds of data that can be collected and the potential bias introduced by the experimenter. This paper expands our understanding of how experimenter roles impact public evaluations and provides an empirical basis to consider different evaluation approaches. We completed an evaluation of a playful gesture-controlled display – not to understand interaction at the display but to compare different evaluation approaches. The conditions placed the experimenter in three roles, steward observer, overt observer, and covert observer, to measure the effect of experimenter presence and analyse the strengths and weaknesses of each approach.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Williamson, Dr Julie and Williamson, Dr John
• Authors: Williamson, J. R., and Williamson, J.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781450346559
• Copyright Holders: Copyright © 2017 ACM
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher