ThyFusionNet: A CNN-transformer framework with spatial aware sparse attention for multi modal thyroid disease diagnosis

In medical image analysis, accurately diagnosing complex lesions remains a formidable challenge, especially for thyroid disorders, which exhibit high incidence and intricate pathology. To enhance diagnostic precision and robustness, we assembled ThyM3, a large-scale multimodal dataset comprising thyroid computed tomography and ultrasound images. Building on this resource, we introduce ThyFusionNet, a novel deep-learning architecture that combines convolutional backbones with transformer modules and performs feature-level fusion to exploit complementary cues across modalities. To improve semantic alignment and spatial modeling, we incorporate head-wise positional encodings and an adaptive sparse attention scheme that suppresses redundant activations while highlighting key features. Skip connections are used to retain low-level details, and a gated-attention fusion block further enriches cross-modal interaction. We also propose an adaptive contrastive-entropy loss that preserves feature consistency and simultaneously enhances prediction discriminability and stability. Extensive experiments demonstrate that ThyFusionNet surpasses current leading methods in accuracy, robustness, and generalization, underscoring its strong potential for clinical deployment.