Creating and augmenting keyboards for extended reality with the Keyboard Augmentation Toolkit

McGill, M., Brewster, S. , De Sa Medeiros, D. P., Bovet, S., Gutierrez, M. and Kehoe, A. (2022) Creating and augmenting keyboards for extended reality with the Keyboard Augmentation Toolkit. ACM Transactions on Computer-Human Interaction, 29(2), 15. (doi: 10.1145/3490495)

Abstract

This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display —2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity —supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts —allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Brewster, Professor Stephen and McGill, Dr Mark
• Authors: McGill, M., Brewster, S., De Sa Medeiros, D. P., Bovet, S., Gutierrez, M., and Kehoe, A.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: ACM Transactions on Computer-Human Interaction
• Publisher: Association for Computing Machinery (ACM)
• ISSN: 1073-0516
• ISSN (Online): 1557-7325
• Copyright Holders: © 2022 Copyright held by the owner/author(s)
• First Published: First published in ACM Transactions on Computer-Human Interaction 29(2):15
• Publisher Policy: Reproduced in accordance with the publisher copyright policy