Coal gangue identification via multispectral image and spectral feature fusion using a multi-scale self-attention residual network

In order to address the issue of poor discrimination between visually similar coal and gangue in images, we propose residual network of multi-scale self-attention (RMSS), a deep-learning method that fuses multispectral image and spectral feature. RMSS builds on a residual network (ResNet) backbone, integrating multi-scale feature fusion (MSFF) and a self-attention mechanism (SAM) to extract image and spectral features, respectively, then fuses them via a multi-input convolutional neural network (CNN). Using five parameters, we selected the 546 nm, 585 nm, and panchromatic bands for our experiments. Experimental results demonstrate that the RMSS model achieves a recognition accuracy of 98.20%. Compared with MobileNetV3, ResNet and EfficientNetV2 models, the RMSS model improves accuracy by 5.75%, 5.39%, and 3.24%, respectively. Under illumination levels from 1000 to 7000lux, incorporating spectral features boosts accuracy by 3–6% with fluctuations within 1.06%. These results indicate that the proposed multispectral fusion method provides a reliable solution for coal gangue identification under complex illumination conditions.