MGRR-Net: Multi-level graph relational reasoning network for facial action unit detection

Ge, X. , Jose, J. M. , Xu, S., Liu, X. and Han, H. (2024) MGRR-Net: Multi-level graph relational reasoning network for facial action unit detection. Transactions on Intelligent Systems and Technology, 15(3), 41. (doi: 10.1145/3643863)

Abstract

The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images, which has attracted extensive research attention due to its wide use in facial expression analysis. Many methods that perform well on automatic facial action unit (AU) detection primarily focus on modeling various AU relations between corresponding local muscle areas or mining global attention–aware facial features; however, they neglect the dynamic interactions among local-global features. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features, as well as the detailed variability across AUs, because of the diversity in expression and individual characteristics. In this article, we propose a novel Multi-level Graph Relational Reasoning Network (termed MGRR-Net) for facial AU detection. Each layer of MGRR-Net performs a multi-level (i.e., region-level, pixel-wise, and channel-wise level) feature learning. On the one hand, the region-level feature learning from the local face patch features via graph neural network can encode the correlation across different AUs. On the other hand, pixel-wise and channel-wise feature learning via graph attention networks (GAT) enhance the discrimination ability of AU features by adaptively recalibrating feature responses of pixels and channels from global face features. The hierarchical fusion strategy combines features from the three levels with gated fusion cells to improve AU discriminative ability. Extensive experiments on DISFA and BP4D AU datasets show that the proposed approach achieves superior performance than the state-of-the-art methods.

• Item Type: Articles
• Additional Information: This research has been supported in part by the National Natural Science Foundation of China (No. 62176249) and in part by the China Scholarship Council (CSC) from the Ministry of Education of China (No.202006310028).
• Keywords: Facial action units, graph attention network, local-global interaction, multi-level relational reasoning.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Jose, Professor Joemon and Ge, Xuri and Xu, Ms Songpei
• Authors: Ge, X., Jose, J. M., Xu, S., Liu, X., and Han, H.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Transactions on Intelligent Systems and Technology
• Publisher: Association for Computing Machinery (ACM)
• ISSN: 2157-6904
• ISSN (Online): 2157-6912
• Copyright Holders: Copyright: © 2024 Copyright held by the owner/author(s)
• First Published: First published in Transactions on Intelligent Systems and Technology 15(3): 41
• Publisher Policy: Reproduced in accordance with the publisher copyright policy