Creating and augmenting keyboards for extended reality with the Keyboard Augmentation Toolkit

McGill, M., Brewster, S. , De Sa Medeiros, D. P., Bovet, S., Gutierrez, M. and Kehoe, A. (2022) Creating and augmenting keyboards for extended reality with the Keyboard Augmentation Toolkit. ACM Transactions on Computer-Human Interaction, 29(2), 15. (doi: 10.1145/3490495)

[img] Text
264087.pdf - Accepted Version

10MB

Abstract

This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display —2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity —supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts —allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.

Item Type:Articles
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Brewster, Professor Stephen and McGill, Dr Mark
Authors: McGill, M., Brewster, S., De Sa Medeiros, D. P., Bovet, S., Gutierrez, M., and Kehoe, A.
College/School:College of Science and Engineering > School of Computing Science
Journal Name:ACM Transactions on Computer-Human Interaction
Publisher:Association for Computing Machinery (ACM)
ISSN:1073-0516
ISSN (Online):1557-7325
Copyright Holders:© 2022 Copyright held by the owner/author(s)
First Published:First published in ACM Transactions on Computer-Human Interaction 29(2):15
Publisher Policy:Reproduced in accordance with the publisher copyright policy

University Staff: Request a correction | Enlighten Editors: Update this record