Bridging behavior and brain: the impact of emotional and linguistic cues on human performance and neural activation.

Human-machine interaction requires seamless integration of multimodal cues to align with users' cognitive and neural states. This study investigates how AI-generated facial expressions, voice tones, and language formality jointly shape cognitive load, fatigue, situation awareness (SA), and neural activation. In a simulated industrial task (N = 60), behavioural results revealed that negative facial expressions combined with formal language elevated cognitive load regardless of voice tones, while informal language paired with negative voice tones and facial expressions maximised fatigue. SA improved when formal language was paired with positive facial expressions or negative voice tones. Neuroimaging identified three-way interactions in premotor cortex, Broca's area, and frontal eye fields, alongside two-way interactions in frontopolar and occipital cortices. Critically, reduced SA correlated with diminished activation in visual and executive control regions. These findings provide neurobehavioral guidelines for designing AI systems that adapt multimodal outputs to mitigate cognitive strain and enhance operational safety.