Breathing Life Into Biomechanical User Models

Ikkala, A., Fischer, F., Klar, M., Bachinski, M., Fleig, A., Howes, A., Hämäläinen, P., Müller, J., Murray-Smith, R. and Oulasvirta, A. (2022) Breathing Life Into Biomechanical User Models. In: UIST ’22: The 35th Annual ACM Symposium on User Interface Software and Technology, Bend, OR, USA, 29 Oct - 02 Nov 2022, p. 90. ISBN 9781450393201 (doi: 10.1145/3526113.3545689)

Abstract

Forward biomechanical simulation in HCI holds great promise as a tool for evaluation, design, and engineering of user interfaces. Although reinforcement learning (RL) has been used to simulate biomechanics in interaction, prior work has relied on unrealistic assumptions about the control problem involved, which limits the plausibility of emerging policies. These assumptions include direct torque actuation as opposed to muscle-based control; direct, privileged access to the external environment, instead of imperfect sensory observations; and lack of interaction with physical input devices. In this paper, we present a new approach for learning muscle-actuated control policies based on perceptual feedback in interaction tasks with physical input devices. This allows modelling of more realistic interaction tasks with cognitively plausible visuomotor control. We show that our simulated user model successfully learns a variety of tasks representing different interaction methods, and that the model exhibits characteristic movement regularities observed in studies of pointing. We provide an open-source implementation which can be extended with further biomechanical models, perception models, and interactive environments.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Murray-Smith, Professor Roderick
• Authors: Ikkala, A., Fischer, F., Klar, M., Bachinski, M., Fleig, A., Howes, A., Hämäläinen, P., Müller, J., Murray-Smith, R., and Oulasvirta, A.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781450393201
• Copyright Holders: Copyright © 2022 The Authors
• First Published: First published in UIST ’22: The 35th Annual ACM Symposium on User Interface Software and Technology: 90
• Publisher Policy: Reproduced under a Creative Commons License

Related URLs

• Organisation

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
300982		Exploiting Closed-Loop Aspects in Computationally and Data Intensive Analytics	Roderick Murray-Smith	Engineering and Physical Sciences Research Council (EPSRC)	EP/R018634/1	Computing Science