Interaction mix and match: synthesizing close interaction using conditional hierarchical GAN with multi-hot class embedding

Goel, A., Men, Q. and Ho, E. S. L. (2022) Interaction mix and match: synthesizing close interaction using conditional hierarchical GAN with multi-hot class embedding. Computer Graphics Forum, 41(8), pp. 327-338. (doi: 10.1111/cgf.14647)

Abstract

Synthesizing multi-character interactions is a challenging task due to the complex and varied interactions between the characters. In particular, precise spatiotemporal alignment between characters is required in generating close interactions such as dancing and fighting. Existing work in generating multi-character interactions focuses on generating a single type of reactive motion for a given sequence which results in a lack of variety of the resultant motions. In this paper, we propose a novel way to create realistic human reactive motions which are not presented in the given dataset by mixing and matching different types of close interactions. We propose a Conditional Hierarchical Generative Adversarial Network with Multi-Hot Class Embedding to generate the Mix and Match reactive motions of the follower from a given motion sequence of the leader. Experiments are conducted on both noisy (depth-based) and high-quality (MoCap-based) interaction datasets. The quantitative and qualitative results show that our approach outperforms the state-of-the-art methods on the given datasets. We also provide an augmented dataset with realistic reactive motions to stimulate future research in this area.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Ho, Dr Edmond S. L
• Authors: Goel, A., Men, Q., and Ho, E. S. L.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Computer Graphics Forum
• Publisher: Wiley
• ISSN: 0167-7055
• ISSN (Online): 1467-8659
• Published Online: 20 March 2023
• Copyright Holders: Copyright © 2023 The Authors
• First Published: First published in Computer Graphics Forum 41(8): 327-338
• Publisher Policy: Reproduced under a Creative Commons License

Related URLs

• Author