A Hybrid ConvNeXtV2–MaxViT Framework with CNN-based Feature Refinement for Skin Lesion Classification

Accurate and prompt detection of skin disorders, particularly malignant skin cancers like melanoma, is considered crucial for effective treatment and improved clinical outcomes. It is a difficult task for even experienced dermatologists to correctly distinguish between similar skin lesions. Deep neural architectures, like Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), can be used for the automated classification of abnormalities in dermoscopic images. ViTs require a large amount of data for optimal generalization, while CNNs are less effective at identifying global patterns. A hybrid model that overcomes the limitations of CNNs and ViTs is proposed in this work. A CNN-based feature refinement module is included in the proposed system to improve lesion-focused features while suppressing irrelevant background information. A dual-path classification algorithm utilizing ConvNeXtV2 for efficient local feature identification and MaxViT to model broader contextual relationships is then employed. The proposed architecture was evaluated on the HAM10000 dataset and validated on the ISIC dataset.  The proposed model outperforms single CNN, ViT, and classical CNN-ViT combination models, based on the experimental results. The architecture discussed here achieves an accuracy of 96.8%, an AUC of 0.978, and a balanced F1-score of 0.965 on the HAM10000 dataset, while demonstrating competitive performance when validated on the ISIC dataset. The effect of CNN-based feature refinement has also been studied. These results demonstrate the effectiveness of combining CNN-based feature refinement with multi-scale feature identification to develop robust and accurate systems for skin disease classification.