How to retrain PeopleNet Transformer with TAO Toolkit?

[masaki_yamagishi]

• Hardware: Geforce RTX4090
• Detectnet_v2
• TLT Version: TAO v 4 or v5

I want to retrain PeopleNet Transformer with TAO Toolkit.
As far as I can see, available one of that model is only provided as ONNX.
I tried to get .tlt file, “trainable_v1.0”, to retrain on TAO v4, but it seem only including pickle file, and I don’t know how I use it.

But, TAO BYOM is only usable to UNET and Classification as below, so I don’t know how I can retrain PeopleNet Transformer with TAO Toolkit.

Cannot convert ONNX model using tao_byom

Yes, for BYOM, currently only the Unet and Classification models are supported by the TAO BYOM Converter. The tool tao-converter is used to convert .tlt model into .etlt model. If the .onnx model is not Unet or Classification, currently it is not workable to convert it to be a pretrained model to be running with TAO. But you can build a TensorRT engine from ONNX using the trtexec tool . See Quick Start Guide :: NVIDIA Deep Learning TensorRT Documentation . But this is not a topic of TAO. It…

Could you tell me some instructions?

[Morganh]

@masaki_yamagishi

I want to retrain PeopleNet Transformer with TAO Toolkit.
As far as I can see, available one of that model is only provided as ONNX.

Please refer to https://forums.developer.nvidia.com/t/training-on-custom-dataset/285501/5?u=morganh.
There are three different kinds of peoplenet now.
One is trained from detectnet_v2 network. (model card is: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet.)
2nd is trained from D-DETR network. (model card is: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet_transformer.)
3rd is trained from DINO network.(model card is: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet_transformer_v2.)

For the 1st peoplenet, you can refer to detectnet_v2 doc and notebook. (DetectNet_v2 - NVIDIA Docs and tao_tutorials/notebooks/tao_launcher_starter_kit/detectnet_v2 at main · NVIDIA/tao_tutorials · GitHub )

For the 2nd peoplenet, you can refer to D-DETR doc and notebook. Deformable DETR - NVIDIA Docs and tao_tutorials/notebooks/tao_launcher_starter_kit/deformable_detr at main · NVIDIA/tao_tutorials · GitHub .

For the 3rd peoplenet, you can refer to DINO doc and notebook. DINO - NVIDIA Docs and tao_tutorials/notebooks/tao_launcher_starter_kit/dino at main · NVIDIA/tao_tutorials · GitHub

[masaki_yamagishi]

Thank you for your reply.

I tried notebook of DINO, and it worked.
It seems the model file (.pth) contains only “data.pkl”, "version"file, and “data” directory.
The model isn’t provided solid file format, such as “.hdf5” like PeopleNet (detectnet_v2), right?

BTW, does BYOM converter still only work for UNET and Classification?
Is there any plan to get compatibility to more general kinds of AI model?

Best regards,

[Morganh]

@masaki_yamagishi

model file (.pth)

Dino network is under tao pytorch docker. The .pth is the pretrained model which can be used to run training. You can use it to finetune your own dataset.
You can retrain PeopleNet Transformer with D-DETR network or DINO network under tao pytorch docker(TAO Toolkit | NVIDIA NGC) .

@masaki_yamagishi

BTW, does BYOM converter still only work for UNET and Classification?
Is there any plan to get compatibility to more general kinds of AI model?

BYOM is used to convert an open-source ONNX model to a TAO-compatible model for running training. Currently it supports Unet and Classification only and there is not other plan yet.

[masaki_yamagishi]

OK, I get it.

Thank you very much for your kindly support.

Best regards,