TAO Toolkit Converter

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc): Nvidia 3090
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc): Detectnet_v2
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here):
Configuration of the TAO Toolkit Instance

task_group:
     model:
        dockers:
            nvidia/tao/tao-toolkit:
                5.0.0-tf2.11.0:
                    docker_registry: nvcr.io
                    tasks:
                        1. classification_tf2
                        2. efficientdet_tf2
                5.0.0-tf1.15.5:
                    docker_registry: nvcr.io
                    tasks:
                        1. bpnet
                        2. classification_tf1
                        3. converter
                        4. detectnet_v2
                        5. dssd
                        6. efficientdet_tf1
                        7. faster_rcnn
                        8. fpenet
                        9. lprnet
                        10. mask_rcnn
                        11. multitask_classification
                        12. retinanet
                        13. ssd
                        14. unet
                        15. yolo_v3
                        16. yolo_v4
                        17. yolo_v4_tiny
                5.2.0-pyt2.1.0:
                    docker_registry: nvcr.io
                    tasks:
                        1. action_recognition
                        2. centerpose
                        3. deformable_detr
                        4. dino
                        5. mal
                        6. ml_recog
                        7. ocdnet
                        8. ocrnet
                        9. optical_inspection
                        10. pointpillars
                        11. pose_classification
                        12. re_identification
                        13. visual_changenet
                5.2.0-pyt1.14.0:
                    docker_registry: nvcr.io
                    tasks:
                        1. classification_pyt
                        2. segformer
    dataset:
        dockers:
            nvidia/tao/tao-toolkit:
                5.2.0-data-services:
                    docker_registry: nvcr.io
                    tasks:
                        1. augmentation
                        2. auto_label
                        3. annotations
                        4. analytics
    deploy:
        dockers:
            nvidia/tao/tao-toolkit:
                5.2.0-deploy:
                    docker_registry: nvcr.io
                    tasks:
                        1. visual_changenet
                        2. centerpose
                        3. classification_pyt
                        4. classification_tf1
                        5. classification_tf2
                        6. deformable_detr
                        7. detectnet_v2
                        8. dino
                        9. dssd
                        10. efficientdet_tf1
                        11. efficientdet_tf2
                        12. faster_rcnn
                        13. lprnet
                        14. mask_rcnn
                        15. ml_recog
                        16. multitask_classification
                        17. ocdnet
                        18. ocrnet
                        19. optical_inspection
                        20. retinanet
                        21. segformer
                        22. ssd
                        23. trtexec
                        24. unet
                        25. yolo_v3
                        26. yolo_v4
                        27. yolo_v4_tiny
format_version: 3.0
toolkit_version: 5.2.0
published_date: 12/06/2023

• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.):
I am experiencing this issue for every model that uses converter so cannot specify it.

If I am right, since tlt 5.0.0, tlt/etlt are deprecated and the toolkit makes models with hdf5 also exports models to onnx format.

But tao-converter requires encrypted model such as etlt format I guess since I am getting this error “[ERROR] Failed to parse the model, please check the encoding key to make sure it’s correct”.

I have tried to use explicit key and trained model several times just in case I used wrong key and none of them worked. While when I used pruned etlt format model from ngc example, everything worked fine. So format is the only issue in my opinion.

Am I experiencing it because I am using too old examples? Should I use trtexec to convert my onnx model instead of using tao-converter?

2

Yes, correct.

Yes, please use trtexec to convert. Refer to Optimizing and Profiling with TensorRT - NVIDIA Docs.

3

I have tested multiple times with the method that I mentioned above.

Also, I have read the doc thoroughly and found that now TAO has gen_trt_engine and you can deploy this to convert onnx to trt model.

Both method made no error except I need to define the batch size (thought trt model converted model from onnx model is not reliable on batch size) but the result is trash.

000000
0000001242×375 110 KB

000005
0000051242×375 108 KB

These are the results from quantized model.

000000 (1)
000000 (1)1242×375 84.6 KB

000005 (1)
000005 (1)1242×375 87.2 KB

These are the results retrained pruned model.

Using trtexec and gen_trt_engine make the same result, so I dont think converting went wrong. Maybe I INT8-optimized the model improperly?

4

Can you share the full command or full steps? Such as, which docker you are using? And which quantized model did you use?

Please share the full command or full steps as well. Thanks.

5

Currently I am using jupyter-lab to run the code. I will just upload the notebook file.

TrafficCamNet.zip (35.2 MB)

if there is anything you need more please let me know and thank you for your help by the way. I’ve read some of your comments on other posts and they are really helpful.

6

For the quantized modelel I will just upload the model that I got from the code.
model_files.zip (3.9 MB)

7

Could you please share the link? Thanks.

8

Since I am running it on local system, I edited my post so you can see check the whole process.

9

OK, your notebook are based on tao_tutorials/notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb at main · NVIDIA/tao_tutorials · GitHub, right?

10

Ehh… No I was using the old version from ngc page. It is out-dated and I was fixing it to use it.

I will check the git you sent me and ask again once I face any issue. Thanks :)

1 Like
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.