ActionRecognitionNet deployment error : Failed to parse onnx model (IIfConditionalOutputLayer)

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello,
I’ve recently trained an action recognition model in tao launcher kit sample notebook using custom data.I’ve successfully exported the model to onnx format using this command:

!tao model action_recognition export \
                   -e $SPECS_DIR/test_i3d_export.yaml \
                   -k $KEY \
                   results_dir=$RESULTS_DIR/rgb_3d_ptm \
                   export.checkpoint=$RESULTS_DIR/rgb_3d_ptm/train/rgb_i3d_128_modelv2.tlt \
                   export.onnx_file=$RESULTS_DIR/export/rgb_i3d_128_model.onnx

I tried to generate an inference engine for the exported model using

!tao deploy trtexec  --onnx=$RESULTS_DIR/export/rgb_i3d_128_model.onnx \
        --maxShapes=input_rgb:16x3x128x224x224 \
        --minShapes=input_rgb:1x3x128x224x224 \
        --optShapes=input_rgb:4x3x128x224x224 \
        --best \
        --saveEngine=$RESULTS_DIR/export/rgb_i3d_128_modelv2.engine \
        --verbose

but I got the following error while parsing the model layers :

[08/30/2023-09:30:56] [V] [TRT] Registering layer: /If_OutputLayer for ONNX node: /If_OutputLayer
[08/30/2023-09:30:56] [E] Error[4]: /If_OutputLayer: IIfConditionalOutputLayer inputs must have the same shape.

I checked the onnx model using the netron app and I found that “then” & “else” branches of the “if” node have different shapes which is currently not supported in tensorrt,I find this error quite strange since I’ve followed Action Recognition documentation as it is and still got this error.
I tried to work around this error by using

onnxsim

Screenshot from 2023-08-30 13-23-14

which simplified the"If" node but the simplified model results are different from the .tlt version and it is no longer reliable.Any suggestions on how I could solve this?

• Hardware: RTX 3060 Driver version 535.86
• Network Type: ActionRecognitionNet
• TLT Version: TAO 5.0.0-deploy
• TensorRT version: 8.5.3 + CUDA 12

3

Could you please share test_i3d_export.yaml ?

4

Here is the spec file you requested:
test_i3d_export.yaml (577 Bytes)
I’m using i3d pretrained weights specified in this repo.

5

Do you mean you were training with “i3d pretrained weights” as pretrained model ? Could you please share the link of this weights? Thanks.

6

Thank you for your answer. Yes,I used the “i3d kinetics pretrained model” found in the repo I mentioned.
Here is the spec file I used for training:
train_rgb_3d_128_i3d.yaml (856 Bytes).

7

To narrow down, please follow TRTEXEC with ActionRecognitionNet - NVIDIA Docs to generate engine with fp16 mode to check if it works.

8

I did try the fp16 precision and still got the same error,the problem seems to be with the model parsing,tensorrt doesn’t support the “If” node having different shapes in its branches. So it is related to the model architecture which is different from the original pretrained model used in the actionrecognitionnet example.

9

We will check further. Thanks for the finding.

10

Based on https://github.com/NVIDIA/tao_pytorch_backend/blob/main/nvidia_tao_pytorch/cv/action_recognition/scripts/export.py, please try below new export.py.
export.py (8.2 KB) .
You can login docker and then run following commands.

$ docker run --runtime=nvidia -it nvcr.io/nvidia/tao/tao-toolkit:5.0.0-pyt /bin/bash
Then inside the docker, 
root@cd9a119e7e02:/opt/nvidia/tools# mv /usr/local/lib/python3.8/dist-packages/nvidia_tao_pytorch/cv/action_recognition/scripts/export.py /usr/local/lib/python3.8/dist-packages/nvidia_tao_pytorch/cv/action_recognition/scripts/export.py.bak
root@cd9a119e7e02:/opt/nvidia/tools# cp export.py /usr/local/lib/python3.8/dist-packages/nvidia_tao_pytorch/cv/action_recognition/scripts/export.py

root@cd9a119e7e02:/opt/nvidia/tools# action_recognition export xxx
11

Thank you for your reply @Morganh , I’ve tried this solution and I’ve successfully generated the engine. It seems that the approach I’ve taken was the right one

The simplified model had nothing to do with the unreliable results, it was the preprocessing configuration in deepstream that affected the output.I’ve tried inferencing directly with the engine in a script and the results were similar to “.tlt” version.
Also, can you confirm that simplifying the onnx model doesn’t affect its performance?
Thank you for your help.

12

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Above-mentioned solution from my last comment is to solve export regression issue due to pytorch upgrade. The squeeze operation in torch leading to If node in ONNX model which cannot be parsed correctly by tensorrt.
For performance, you can run trtexec to check the fps.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.