Real-time hand interaction and self-directed machine learning agents in immersive learning environments

Iqbal, M. Z. and Campbell, A. G. (2023) Real-time hand interaction and self-directed machine learning agents in immersive learning environments. Computers and Education: X Reality, 3, 100038. (doi: 10.1016/j.cexr.2023.100038)

Abstract

Integration of extended reality (XR) in education is becoming popular to transform the traditional classroom with immersive learning environments. The adoption of immersive learning is accelerating as an innovative approach for science and engineering subjects. With new powerful interaction techniques in XR and the latest developments in artificial intelligence, interactive and self-directed learning are becoming important. However, there is a lack of research exploring these emerging technologies research with kinesthetic learning or “hands-one learning" as a pedagogical approach using real-time hand interaction and agent-guided learning in immersive environments. This paper proposes a novel approach that uses machine learning agents to facilitate interactive kinesthetic learning in science and engineering education through real-time hand interaction in the virtual world. To implement the following approach, this paper uses a chemistry-related case study and presents a usability evaluation conducted with 15 expert reviewers and 2 subject experts. NASA task load index is used for cognitive workload measurement, and the technology acceptance model is used for measuring perceived ease of use and perceived usefulness in the evaluations. The evaluation with expert reviewers proposed self-directed learning using trained agents can help in the end-user training in learning technical topics and controller-free hand interaction for kinesthetic tasks can improve hands-on learning motivation in virtual laboratories. This success points to a novel research area where agents embodied in an immersive environment using machine learning techniques can forge a new pedagogical approach where they can act as both teacher and assessor.

• Item Type: Articles
• Keywords: Mixed reality, virtual reality, extended reality, technology enhanced learning, immersive learning, XR, learning technology
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Iqbal, Dr Zahid
• Authors: Iqbal, M. Z., and Campbell, A. G.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Computers and Education: X Reality
• Journal Abbr.: CEXR
• Publisher: Elsevier
• ISSN: 2949-6780
• ISSN (Online): 2949-6780
• Published Online: 07 September 2023
• Copyright Holders: Copyright © 2023 The Authors
• First Published: First published in Computers & Education: X Reality 3:100038
• Publisher Policy: Reproduced under a Creative Commons License