Knowledge Distillation

Hello everyone.
To your knowledge, is it possible to perform some form of knowledge distillation on the onnx models for the object detection task on the Jetson Nano?
Thank you.

Hi,

You can apply this on a desktop GPU.
And run the smaller model on the Nano directly.

Thanks.

Thanks for the reply @AastaLLL .
I had thought about creating and training both the teacher and the student on Colab (having an Apple compouter). I intend to use the https://nvidia.box.com/shared/static/djf5w54rjvpqocsiztzaandq1m3avr7c.pth model as a teacher model (available for download in the “Re-training SSD-Mobilenet” guide). Unfortunately, being an SSD model, I don’t think this has a softmax layer from which to take the soft labels useful to the student. Do you have any guide who can do that?

Thank you.

Hi,

There are some researches related to the detection model distillation.
You can check it and find some ideas:

Data-free Knowledge Distillation for Object Detection

Knowledge Distillation for Object Detection via Rank Mimicking and
Prediction-guided Feature Imitation

Thanks.