How to use pretrained detector weights only for specific target classes in TAO?

[dhxh101]

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) : Ubuntu20.04, x86, Quadro RTX 4000
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) : Detectnet_v2
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) : nvcr.io/nvidia/tao/tao-toolkit:4.0.0-tf1.15.5
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I am trying to get models detecting multiple classes including “person” by retraining pretrained PeopleNet.
Here is the pretrained model file:
https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplenet/versions/trainable_v2.6/files/resnet34_peoplenet.tlt

I want to initialize detector weights for “person” and the other target classes with pretrained and random ones, respectively.

Detection network as well as feature extraction network can be initialized with pretrained ones by setting “load_graph” to true in the training spec file, but I guess in this case the number of target classes must be three (corresponding to “person”, “bag”, and “face”) and detector weights other than “person” cannot be randomly initialized.

I would like to know any ways to use pretrained detector weights only for specific target classes.

[Morganh]

While using pretrained model, currently by default, we cannot use pretrained detector weights only for specific target classes. The weights for bag and face will be also initialized.

[dhxh101]

Thanks for the replay.
Is there any plan to support this feature in future TAO releases?

[Morganh]

Need to sync internally. And you can also modify code to meet custom requirement. Currently, the detecnet_v2 network code is in https://github.com/NVIDIA/tao_tensorflow1_backend/tree/main/nvidia_tao_tf1/cv/detectnet_v2 .

[dhxh101]

Thank you. I understand the situation.
I’ll look into the code.