Knowledge Distillation

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hello everyone.
To your knowledge, is it possible to perform some form of knowledge distillation on the onnx models for the object detection task on the Jetson Nano?
Thank you.

3

Hi,

You can apply this on a desktop GPU.
And run the smaller model on the Nano directly.

Thanks.

4

Thanks for the reply @AastaLLL .
I had thought about creating and training both the teacher and the student on Colab (having an Apple compouter). I intend to use the https://nvidia.box.com/shared/static/djf5w54rjvpqocsiztzaandq1m3avr7c.pth model as a teacher model (available for download in the “Re-training SSD-Mobilenet” guide). Unfortunately, being an SSD model, I don’t think this has a softmax layer from which to take the soft labels useful to the student. Do you have any guide who can do that?

Thank you.

5

Hi,

There are some researches related to the detection model distillation.
You can check it and find some ideas:

Data-free Knowledge Distillation for Object Detection

Knowledge Distillation for Object Detection via Rank Mimicking and
Prediction-guided Feature Imitation

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.