Hybrid Soft-Label and Feature-Space Distillation for Traditional Chinese Medicine Imagery

Driven by the rapid advancement of artificial intelligence technologies, deep learning has achieved significant success in image classification domains. However, deploying large-scale pre-trained models on resource-constrained edge devices remains challenging due to their massive parameter counts and substantial computational and storage overhead across. To address the inherent conflict between accuracy and efficiency in Traditional Chinese Medicine (TCM) image classification tasks, this paper proposes a novel TCM classification method based on a dual-knowledge distillation framework. This approach leverages high-performance ViT-Large and ResNet152 architectures as teacher models, transferring multi-level knowledge to lightweight student networks (MobileNetV3 and EfficientNet-B0) via a combination of output-level soft label distillation and intermediate feature layer distillation. Extensive experiments conducted on a constructed 12-category TCM dataset demonstrate that this dual mechanism significantly enhances student model performance. Compared to baselines trained from scratch, the distilled MobileNetV3 and EfficientNet-B0 achieved substantial Top-1 accuracy gains of 8.3% and 7.6% respectively, reaching a peak accuracy of 91.8% and significantly bridging the performance gap towards teacher models. Ablation studies further confirm the complementary nature and effectiveness of the two distillation strategies. Furthermore, the optimized models retain a minimal parameter footprint while achieving high accuracy, maintaining mobile inference latencies consistently below 100ms, thereby demonstrating high deployability and practical application value. Finally, this paper discusses the impact of hyperparameters and outlines future optimization directions such as multimodal fusion.