Converting TAO-trained MaskRCNN models to ONNX for CPU inference

Accelerated Computing Intelligent Video Analytics TAO Toolkit
, ,
1

• Hardware (RTX 3090)
• Network Type (Mask_rcnn)

Hi, I want to perform CPU inference using TAO-trained MaskRCNN models. I previously was able to do so using TAO-trained DetectNet_v2 models by converting them into ONNX, and I was hoping to do the same for TAO-trained MaskRCNN models.

I’m trying to do this as I want to use the ONNX weights with a CPU-only instance of Triton Inference Server. It currently works with the TAO-trained DetectNet_v2-converted ONNX models, and I’m trying to figure out how to do so for the TAO-trained MaskRCNN models as well.

I understand that back in 2022, there was no support for direct TAO-trained MaskRCNN → ONNX conversion. Does this still hold true as of today?

If yes, are there other, perhaps indirect ways to perform this conversion? (E.g. converting/exporting into an intermediary file format that can then be converted into ONNX?)

3

Since TAO 5.0, you can generate the .onnx file instead of .etlt file.
You can refer to 5.0 or latest 5.5(link) notebook.

4

Using TAO 5.3.0, upon running the relevant command (10. Deploy!):

!tao model mask_rcnn export -m $USER_EXPERIMENT_DIR/experiment_dir_retrain/model.epoch-$NUM_EPOCH.tlt \
                      -e $SPECS_DIR/maskrcnn_retrain_resnet50.txt \
                      -k tlt_encode \
                      --gen_ds_config

I get an UFF file instead of an ONNX file. Specifically, the logs show:

⋮ 
2024-10-02 04:12:25,307 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.mask_rcnn.export.exporter 300: Converted model was saved into /usr/src/app/mask_rcnn/experiment_dir_unpruned/model.epoch-199.uff
⋮

Is this expected?

5

After checking, currently, Mask_rcnn does not support export to ONNX file yet. Suggest you to use trtexec to generate tensorrt engine for running inference on GPU.

6

I see. For my use case:

does that mean that there is no way at all for me to get TAO-trained MaskRCNN models running for CPU-only instances?

7

If no, is export to ONNX on the roadmap for the next update/release?

8

Will sync internally for this feature request.