Local Global Relational Network for Facial Action Units Recognition

Ge, X., Wang, P., Han, H., Jose, J. M. , Ji, Z., Wu, Z. and Liu, X. (2021) Local Global Relational Network for Facial Action Units Recognition. In: 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), Jodhpur, India, 15-18 Dec 2021, ISBN 9781665431774 (doi: 10.1109/FG52635.2021.9666961)

Abstract

Many existing facial action units (AUs) recognition approaches often enhance the AU representation by combining local features from multiple independent branches, each corresponding to a different AU. However, such multi-branch combination-based methods usually neglect potential mutual assistance and exclusion relationship between AU branches or simply employ a pre-defined and fixed knowledge-graph as a prior. In addition, extracting features from pre-defined AU regions of regular shapes limits the representation ability. In this paper, we propose a novel Local Global Relational Network (LGRNet) for facial AU recognition. LGRNet mainly consists of two novel structures, i.e., a skip-BiLSTM module which models the latent mutual assistance and exclusion relationship among local AU features from multiple branches to enhance the feature robustness, and a feature fusion&refining module which explores the complementarity between local AUs and the whole face in order to refine the local AU features to improve the discriminability. Experiments on the BP4D and DISFA AU datasets show that the proposed approach outperforms the state-of-the-art methods by a large margin.

• Item Type: Conference Proceedings
• Additional Information: This work was supported in part by National Key R&D Program of China (No. 2020AAA0104500), China Scholarship Council (CSC) from the Ministry of Education of P.R. China (No. 202006310028), Natural Science Foundation of China (grants 61732004 and 61672496), and Youth Innovation Promotion Association CAS (grant 2018135).
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Jose, Professor Joemon and Ge, Ms Xuri
• Authors: Ge, X., Wang, P., Han, H., Jose, J. M., Ji, Z., Wu, Z., and Liu, X.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781665431774
• Published Online: 12 January 2022
• Copyright Holders: Copyright © 2022 IEEE
• First Published: First published in Proceedings of the 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021)
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher

Related URLs

• Organisation