Tensorrt 6.0.1 Parametric ReLU

Hi i have converted a model from pytorch to onnx with no error, but when i try to convert the onnx version to tensorRT i got this error

[TensorRT] ERROR: (Unnamed Layer* 3) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
In node 3 (importMaxPool): INVALID_NODE: Assertion failed: dims.nbDims >= 2

Could someone please help me to solve this?

Thanks

Hi,
Can you provide the following information so we can better help?
Provide details on the platforms you are using:
o Linux distro and version
o GPU type
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow version
o TensorRT version
o If Jetson, OS, hw versions

Also, if possible could you please share the model file along with the conversion scripts?

Thanks

Hi,
Thanks for your reply.
I have ubuntu 16.04 Nvidia driver 410 cuda version 10.0. Cudnn version 7.5. Pytorch version 1.1 Tensorrt 6 is on a different machine with ubuntu 18.04 cuda 10.1 cudnn 7.6.3 . I have followed the guide at this link https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html.

The onnx model is ok because i have checked it with Pytorch. When i’am trying to convert it in Tensorrt on the second machine i have the above error.

At this link you can download the model https://drive.google.com/open?id=1DH0tBJk3KrSOeTAKutswOgIBYAUfo5BV

Thanks

Hi,
Thanks for the info. Will look into it.
Meanwhile, can you try converting your model using “trtexec” command?

https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec#example-4-running-an-onnx-model-with-full-dimensions-and-dynamic-shapes

Thanks

Hi,

Your model does not follow the ONNX spec: The PRelu node needs to have it’s slopes input to follow unidirectional broadcasting rules.
https://github.com/onnx/onnx/blob/master/docs/Operators.md#PRelu
https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md#unidirectional-broadcasting

The inputs into the node are the data tensor (size [32,64,122,122]) and slopes (size [64]). Slopes is not broadcastable out of the box. You need to update slope tensors to be size [64,1,1]

Thanks

1 Like

Hi,

thanks i will try to change the model and let you know.

If you have time can you explain me how have you debug this error? There are some tools to check if onnx model in not working. I’m using standard pytorch function to verify if the model is ok, but i haven’t receive any type of error from converting it.

Thanks

Hi,

You can use “trtexec” command line tool to understand performance and possibly locate bottlenecks.

Please find the below links for your reference:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#trtexec
https://github.com/NVIDIA/TensorRT/blob/release/6.0/samples/opensource/trtexec/README.md

Thanks

@eric, I have met the same problem, have you solved it?

Thanks.

Hi,

i haven’t resolved it yet. But it’s a problem with my pytorch model because with mxnet i am able to convert the model with Prelu. As soon as i find a solution i will share it.

Thanks,
Eric

i also meet onnx to trt prelu layer problem in jetpack4.3.
i have tried ./trtexec with command line as below:

./trtexec --onnx=/home/ubuntu/temp_onnx/face_partial_upload.onnx --saveEngine=/home/ubuntu/temp_onnx/face_partial_upload.trt --fp16 --shapes=input:1x3x112x112 --verbose --exportTimes=~/temp_onnx/times.json

[1x3x112x112] is exactly the model input shape, but trtexec FAILED with output:

&&&& RUNNING TensorRT.trtexec # ./trtexec --onnx=/home/ubuntu/temp_onnx/face_partial_upload.onnx --saveEngine=/home/ubuntu/temp_onnx/face_partial_upload.trt --fp16 --shapes=input:1x3x112x112 --verbose --exportTimes=~/temp_onnx/times.json
[11/20/2019-17:36:31] [I] === Model Options ===
[11/20/2019-17:36:31] [I] Format: ONNX
[11/20/2019-17:36:31] [I] Model: /home/ubuntu/temp_onnx/face_partial_upload.onnx
[11/20/2019-17:36:31] [I] Output:
[11/20/2019-17:36:31] [I] === Build Options ===
[11/20/2019-17:36:31] [I] Max batch: explicit
[11/20/2019-17:36:31] [I] Workspace: 16 MB
[11/20/2019-17:36:31] [I] minTiming: 1
[11/20/2019-17:36:31] [I] avgTiming: 8
[11/20/2019-17:36:31] [I] Precision: FP16
[11/20/2019-17:36:31] [I] Calibration: 
[11/20/2019-17:36:31] [I] Safe mode: Disabled
[11/20/2019-17:36:31] [I] Save engine: /home/ubuntu/temp_onnx/face_partial_upload.trt
[11/20/2019-17:36:31] [I] Load engine: 
[11/20/2019-17:36:31] [I] Inputs format: fp32:CHW
[11/20/2019-17:36:31] [I] Outputs format: fp32:CHW
[11/20/2019-17:36:31] [I] Input build shape: input=1x3x112x112+1x3x112x112+1x3x112x112
[11/20/2019-17:36:31] [I] === System Options ===
[11/20/2019-17:36:31] [I] Device: 0
[11/20/2019-17:36:31] [I] DLACore: 
[11/20/2019-17:36:31] [I] Plugins:
[11/20/2019-17:36:31] [I] === Inference Options ===
[11/20/2019-17:36:31] [I] Batch: Explicit
[11/20/2019-17:36:31] [I] Iterations: 10 (200 ms warm up)
[11/20/2019-17:36:31] [I] Duration: 10s
[11/20/2019-17:36:31] [I] Sleep time: 0ms
[11/20/2019-17:36:31] [I] Streams: 1
[11/20/2019-17:36:31] [I] Spin-wait: Disabled
[11/20/2019-17:36:31] [I] Multithreading: Enabled
[11/20/2019-17:36:31] [I] CUDA Graph: Disabled
[11/20/2019-17:36:31] [I] Skip inference: Disabled
[11/20/2019-17:36:31] [I] === Reporting Options ===
[11/20/2019-17:36:31] [I] Verbose: Enabled
[11/20/2019-17:36:31] [I] Averages: 10 inferences
[11/20/2019-17:36:31] [I] Percentile: 99
[11/20/2019-17:36:31] [I] Dump output: Disabled
[11/20/2019-17:36:31] [I] Profile: Disabled
[11/20/2019-17:36:31] [I] Export timing to JSON file: ~/temp_onnx/times.json
[11/20/2019-17:36:31] [I] Export profile to JSON file: 
[11/20/2019-17:36:31] [I] 
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - GridAnchor_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - GridAnchorRect_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - NMS_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - Reorg_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - Region_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - Clip_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - LReLU_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - PriorBox_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - Normalize_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - RPROI_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - BatchedNMS_TRT
[11/20/2019-17:36:31] [V] [TRT] Plugin Creator registration succeeded - FlattenConcat_TRT
----------------------------------------------------------------
Input filename:   /home/ubuntu/temp_onnx/face_partial_upload.onnx
ONNX IR version:  0.0.3
Opset version:    9
Producer name:    pytorch
Producer version: 0.4
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[11/20/2019-17:36:33] [V] [TRT] /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:773: Convolution input dimensions: (3, 112, 112)
[11/20/2019-17:36:33] [E] [TRT] (Unnamed Layer* 0) [Convolution]: at least 4 dimensions are required for input
[11/20/2019-17:36:33] [V] [TRT] /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:840: Using kernel: (3, 3), strides: (2, 2), padding: (1, 1), dilations: (1, 1), numOutputs: 64
[11/20/2019-17:36:33] [V] [TRT] /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:841: Convolution output dimensions: (0)
[11/20/2019-17:36:33] [V] [TRT] 8:Conv -> 
While parsing node number 1 [BatchNormalization -> "9"]:
--- Begin node ---
input: "8"
input: "2"
input: "3"
input: "4"
input: "5"
output: "9"
op_type: "BatchNormalization"
attribute {
  name: "epsilon"
  f: 1e-05
  type: FLOAT
}
attribute {
  name: "momentum"
  f: 1
  type: FLOAT
}
doc_string: "/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/functional.py(1623): batch_norm\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py(76): forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/bokyliu/dukto/fxp/AirFace/2d_facerecognition/20191119-1/test/airFace.py(61): forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/bokyliu/dukto/fxp/AirFace/2d_facerecognition/20191119-1/test/airFace.py(145): forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/jit/__init__.py(252): forward\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/nn/modules/module.py(489): __call__\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/jit/__init__.py(197): get_trace_graph\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/onnx/utils.py(192): _trace_and_get_graph_from_model\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/onnx/utils.py(224): _model_to_graph\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/onnx/utils.py(281): _export\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/onnx/utils.py(104): export\n/home/bokyliu/anaconda3/envs/pytorch1/lib/python3.6/site-packages/torch/onnx/__init__.py(27): export\n/home/bokyliu/dukto/fxp/AirFace/2d_facerecognition/20191119-1/test/pytorch2onnx.py(85): <module>\n"

--- End node ---
ERROR: builtin_op_importers.cpp:695 In function importBatchNormalization:
[6] Assertion failed: scale_weights.shape == weights_shape
[11/20/2019-17:36:33] [E] Failed to parse onnx file
[11/20/2019-17:36:33] [E] Parsing model failed
[11/20/2019-17:36:33] [E] Engine could not be created
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=/home/ubuntu/temp_onnx/face_partial_upload.onnx --saveEngine=/home/ubuntu/temp_onnx/face_partial_upload.trt --fp16 --shapes=input:1x3x112x112 --verbose --exportTimes=~/temp_onnx/times.json

Hi,

As per the above logs, issue doesn’t seems to be prelu layer problem.
It seems to be due to shape mismatch in the BatchNormalization layer.

Please refer to the below link:
https://github.com/onnx/onnx/blob/master/docs/Operators.md#batchnormalization

Could you please share the model file and steps to reproduce the error so we can better help?

Thanks

More detail on github:https://github.com/BokyLiu/Jetpack4.3-Trt6-PReLU-bug-report/tree/master

Hi,

In this case, you need to make sure your PReLU op (the slope) follows the rules here for unidirectional broadcasting:

https://github.com/onnx/onnx/blob/master/docs/Operators.md#prelu
https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md#unidirectional-broadcasting

Thanks