How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling?

Putkonen, A., Nioche, A. , Tanskanen, V., Klami, A. and Oulasvirta, A. (2022) How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling? In: UMAP '22: 30th ACM, Barcelona, Spain, July 4-7 2022, pp. 179-190. (doi: 10.1145/3503252.3531322)

Abstract

Theory-based, or “white-box,” models come with a major benefit that makes them appealing for deployment in user modeling: their parameters are interpretable. However, most theory-based models have been developed in controlled settings, in which researchers determine the experimental design. In contrast, real-world application of these models demands setups that are beyond developer control. In non-experimental, naturalistic settings, the tasks with which users are presented may be very limited, and it is not clear that model parameters can be reliably inferred. This paper describes a technique for assessing whether a naturalistic dataset is suitable for use with a theory-based model. The proposed parameter recovery technique can warn against possible over-confidence in inferred model parameters. This technique also can be used to study conditions under which parameter inference is feasible. The method is demonstrated for two models of decision-making under risk with naturalistic data from a turn-based game.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Nioche, Dr Aurelien
• Authors: Putkonen, A., Nioche, A., Tanskanen, V., Klami, A., and Oulasvirta, A.
• College/School: College of Science and Engineering > School of Computing Science
• Copyright Holders: Copyright © 2022. Copyright held by the owner/author(s)
• First Published: First published in UMAP '22: Proceedings of the 30th ACM
• Publisher Policy: Reproduced under a Creative Commons licence