ConvNeXt knowledge distillation optimization method for SPOTS-10 animal pattern recognition

Animal pattern recognition from nighttime grayscale images is crucial for wildlife protection and ecological monitoring. Most of the current models suffer from a large parameter scale, making them unsuitable for deployment in resource-constrained environments. To address this challenge, this study proposes a multi-layer knowledge distillation approach based on the teacher Convnext model to improve the lightweight student model's classification performance effectively. Experimental results show that the parameters of the distilled CifarResNet20 model are only 0.27M, and the accuracy is 88.76%, which is superior to the traditional single-layer distillation and another tiny student model. The study confirms the efficiency and practical value of the proposed method in practical applications such as ecological monitoring.