Understanding Public Evaluation: Quantifying Experimenter Intervention

Williamson, J. R. and Williamson, J. (2017) Understanding Public Evaluation: Quantifying Experimenter Intervention. In: Proceedings of ACM SIGCHI 2017, Denver, CO, USA, 6-11 May 2017, pp. 3414-3425. ISBN 9781450346559 (doi: 10.1145/3025453.3025598)

135020.pdf - Accepted Version



Public evaluations are popular because some research questions can only be answered by turning “to the wild.” Different approaches place experimenters in different roles during deployment, which has implications for the kinds of data that can be collected and the potential bias introduced by the experimenter. This paper expands our understanding of how experimenter roles impact public evaluations and provides an empirical basis to consider different evaluation approaches. We completed an evaluation of a playful gesture-controlled display – not to understand interaction at the display but to compare different evaluation approaches. The conditions placed the experimenter in three roles, steward observer, overt observer, and covert observer, to measure the effect of experimenter presence and analyse the strengths and weaknesses of each approach.

Item Type:Conference Proceedings
Glasgow Author(s) Enlighten ID:Williamson, Dr Julie and Williamson, Dr John
Authors: Williamson, J. R., and Williamson, J.
College/School:College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright © 2017 ACM
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record