ME-NAS: A Micro Expression Feature Adaptive Neural Architecture Search

Convolution neural networks (CNN) have emerged as a prevailing paradigm for micro-expression recognition (MER) yet, it is inefficient and time-intensive to design optimal CNN-based MER models manually. In recent times, the neural architecture search (NAS) has garnered attention due to its automatic CNN architecture searching ability. However, the performance of NAS in MER is limited by challenges such as rapid duration, subtle intensity, and a mismatch between architecture and cell-level search. The existing search space, which stacks twelve cells with three transition paths (downsample, upsample, and same resolution), creates deep networks that may diminish minute spatio-temporal features due to progressive convolution and pooling. Therefore, motivated by these factors, in this paper, we introduces a novel approach, the Micro-Expression Feature Adaptive Neural Architecture Search (ME-NAS), to analyze true human emotions through micro-expression recognition. While Neural Architecture Search (NAS) has gained attention for its automatic Convolutional Neural Network (CNN) architecture search ability, its application in Micro-Expression Recognition (MER) faces challenges due to ingrained challenges (rapid duration, subtle and low intensity) and the discrepancy between architecture and cell-level search. The existing NAS architecture search space is designed by stacking twelve cells with three transition paths (downsample, upsample, and same resolution), resulting in a deep network. Such deep networks may diminish minute spatiotemporal features due to the progressive convolution and pooling operations. Motivated by these factors, we designed a new NAS algorithm: ME-NAS. The ME-NAS comprises expressive feature exploration (EXPERT) in architecture search along with refined and complementary feature derivative (ReCODE) operations in cell-level search. The EXPERT aims to trace the optimal paths instead of covering all possible paths between cells. The ReCODE operations capture micro-level variations from spatial and temporal domains by introducing 24 3D-convolution operations. The proposed ReCODE and EXPERT search space jointly lead to the search for a robust and shallow CNN architecture for micro expressions (MEs). The proposed ME-NAS is evaluated on six datasets: CASME-I, CASME-II, CAS(ME)2, SAMM, SMIC, and MEGC-19 composite, with two evaluation strategies: LOSO, and cross-domain, respectively. The experimental results manifest that the proposed ME-NAS outperformed the state-of-the-art approaches on both evaluation strategies.