研究背景与实验设置
Diabetic retinopathy grading from retinal fundus photographs is important but difficult to deploy on resource-constrained hardware. We conducted a controlled study of offline response-based knowledge distillation for five-class diabetic retinopathy grading on APTOS 2019, comparing ResNet-50, MobileNetV3-Large, and Swin Transformer-Tiny (Swin-T) teachers distilled into a MobileNetV3-Small student.
- A 3 x 4 x 2 grid of teacher, temperature, and hard-label weight was screened at one seed,
- while six configurations and the no-distillation baseline were evaluated across three seeds.
实验结果分析
The highest observed single-seed quadratic weighted kappa was 0.9093 (macro-F1 0.6851) for Swin-T at temperature 4 and hard-label weight 0.7; the corresponding three-seed mean was 0.9024, compared with 0.8804 for the baseline.
The student used 1.52 M parameters and 5.94 MB, representing an 18.07x parameter reduction relative to Swin-T.
结论与未来展望
Temperature and hard-label weight effects were teacher- and configuration-dependent, while