/builder/cudnnBuilderGraph.cpp (780) - Assertion Error in checkDimsSanity: 0

orcdnz · May 31, 2021, 2:50pm

I’ve been trying to create a YOLO trt engine. But I keep getting

../builder/cudnnBuilderGraph.cpp:780
Aborting...

../builder/cudnnBuilderGraph.cpp (780) - Assertion Error in checkDimsSanity: 0 (dims.d[i] >= 1)

Running builder as;

 engine_ = unique_ptr<nvinfer1::ICudaEngine>(builder->buildEngineWithConfig(*network, *config));

My code is from https://github.com/tier4/AutowareArchitectureProposal.iv/tree/ros2/perception/object_recognition/detection/tensorrt_yolo. It is a ros package but parts for TRT is pretty isolated from ROS. The onnx i use is also here.

spolisetty · June 1, 2021, 6:32am

Hi @orcdnz,

We recommend you to share complete error logs. Please try trtexec and share us verbose logs for better debugging.
Also please share environment details you’re using.
TensorRT Version :
GPU Type :
Nvidia Driver Version :
CUDA Version :
CUDNN Version :
Operating System + Version :
Python Version (if applicable) :
TensorFlow Version (if applicable) :
PyTorch Version (if applicable) :
Baremetal or Container (if container which image + tag) :

For your reference,
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

Thank you.

orcdnz · June 1, 2021, 7:59am

Trtexec manages to run model succesfully with ./trtexec --onnx=yolov3.onnx command.

TRT Version: 7.0.0.11
GPU Type: RTX 2070 Mobile
Nvidia Driver: 450.119.03
CUDA: 10
CUDNN: 7.6.5

spolisetty · June 1, 2021, 8:42am

@orcdnz, looks like you’ve not shared error logs. We request you to please share error logs for better assistance.

Thank you.

orcdnz · June 1, 2021, 8:58am

There is no log about the error. However whole output to terminal is as below.

ImporterContext.hpp:97: Registering tensor: 091_convolutional for ONNX tensor: 091_convolutional
ModelImporter.cpp:180: 091_convolutional [Conv] outputs: [091_convolutional -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 091_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 091_convolutional
ModelImporter.cpp:123: Searching for input: 091_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 091_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 091_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 091_convolutional_bn_var
ModelImporter.cpp:129: 091_convolutional_bn [BatchNormalization] inputs: [091_convolutional -> (1, 512, 38, 38)], [091_convolutional_bn_scale -> (512)], [091_convolutional_bn_bias -> (512)], [091_convolutional_bn_mean -> (512)], [091_convolutional_bn_var -> (512)], 
ImporterContext.hpp:122: Registering layer: 091_convolutional_bn for ONNX node: 091_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 091_convolutional_bn for ONNX tensor: 091_convolutional_bn
ModelImporter.cpp:180: 091_convolutional_bn [BatchNormalization] outputs: [091_convolutional_bn -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 091_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 091_convolutional_bn
ModelImporter.cpp:129: 091_convolutional_lrelu [LeakyRelu] inputs: [091_convolutional_bn -> (1, 512, 38, 38)], 
ImporterContext.hpp:122: Registering layer: 091_convolutional_lrelu for ONNX node: 091_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 091_convolutional_lrelu for ONNX tensor: 091_convolutional_lrelu
ModelImporter.cpp:180: 091_convolutional_lrelu [LeakyRelu] outputs: [091_convolutional_lrelu -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 092_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 091_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 092_convolutional_conv_weights
ModelImporter.cpp:129: 092_convolutional [Conv] inputs: [091_convolutional_lrelu -> (1, 512, 38, 38)], [092_convolutional_conv_weights -> (256, 512, 1, 1)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 512, 38, 38)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 256
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 256, 38, 38)
ImporterContext.hpp:122: Registering layer: 092_convolutional for ONNX node: 092_convolutional
ImporterContext.hpp:97: Registering tensor: 092_convolutional for ONNX tensor: 092_convolutional
ModelImporter.cpp:180: 092_convolutional [Conv] outputs: [092_convolutional -> (1, 256, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 092_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 092_convolutional
ModelImporter.cpp:123: Searching for input: 092_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 092_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 092_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 092_convolutional_bn_var
ModelImporter.cpp:129: 092_convolutional_bn [BatchNormalization] inputs: [092_convolutional -> (1, 256, 38, 38)], [092_convolutional_bn_scale -> (256)], [092_convolutional_bn_bias -> (256)], [092_convolutional_bn_mean -> (256)], [092_convolutional_bn_var -> (256)], 
ImporterContext.hpp:122: Registering layer: 092_convolutional_bn for ONNX node: 092_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 092_convolutional_bn for ONNX tensor: 092_convolutional_bn
ModelImporter.cpp:180: 092_convolutional_bn [BatchNormalization] outputs: [092_convolutional_bn -> (1, 256, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 092_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 092_convolutional_bn
ModelImporter.cpp:129: 092_convolutional_lrelu [LeakyRelu] inputs: [092_convolutional_bn -> (1, 256, 38, 38)], 
ImporterContext.hpp:122: Registering layer: 092_convolutional_lrelu for ONNX node: 092_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 092_convolutional_lrelu for ONNX tensor: 092_convolutional_lrelu
ModelImporter.cpp:180: 092_convolutional_lrelu [LeakyRelu] outputs: [092_convolutional_lrelu -> (1, 256, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 093_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 092_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 093_convolutional_conv_weights
ModelImporter.cpp:129: 093_convolutional [Conv] inputs: [092_convolutional_lrelu -> (1, 256, 38, 38)], [093_convolutional_conv_weights -> (512, 256, 3, 3)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 256, 38, 38)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 512
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 512, 38, 38)
ImporterContext.hpp:122: Registering layer: 093_convolutional for ONNX node: 093_convolutional
ImporterContext.hpp:97: Registering tensor: 093_convolutional for ONNX tensor: 093_convolutional
ModelImporter.cpp:180: 093_convolutional [Conv] outputs: [093_convolutional -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 093_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 093_convolutional
ModelImporter.cpp:123: Searching for input: 093_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 093_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 093_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 093_convolutional_bn_var
ModelImporter.cpp:129: 093_convolutional_bn [BatchNormalization] inputs: [093_convolutional -> (1, 512, 38, 38)], [093_convolutional_bn_scale -> (512)], [093_convolutional_bn_bias -> (512)], [093_convolutional_bn_mean -> (512)], [093_convolutional_bn_var -> (512)], 
ImporterContext.hpp:122: Registering layer: 093_convolutional_bn for ONNX node: 093_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 093_convolutional_bn for ONNX tensor: 093_convolutional_bn
ModelImporter.cpp:180: 093_convolutional_bn [BatchNormalization] outputs: [093_convolutional_bn -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 093_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 093_convolutional_bn
ModelImporter.cpp:129: 093_convolutional_lrelu [LeakyRelu] inputs: [093_convolutional_bn -> (1, 512, 38, 38)], 
ImporterContext.hpp:122: Registering layer: 093_convolutional_lrelu for ONNX node: 093_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 093_convolutional_lrelu for ONNX tensor: 093_convolutional_lrelu
ModelImporter.cpp:180: 093_convolutional_lrelu [LeakyRelu] outputs: [093_convolutional_lrelu -> (1, 512, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 094_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 093_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 094_convolutional_conv_weights
ModelImporter.cpp:123: Searching for input: 094_convolutional_conv_bias
ModelImporter.cpp:129: 094_convolutional [Conv] inputs: [093_convolutional_lrelu -> (1, 512, 38, 38)], [094_convolutional_conv_weights -> (255, 512, 1, 1)], [094_convolutional_conv_bias -> (255)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 512, 38, 38)
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 255
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 255, 38, 38)
ImporterContext.hpp:122: Registering layer: 094_convolutional for ONNX node: 094_convolutional
ImporterContext.hpp:97: Registering tensor: 094_convolutional_1 for ONNX tensor: 094_convolutional
ModelImporter.cpp:180: 094_convolutional [Conv] outputs: [094_convolutional -> (1, 255, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 097_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 092_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 097_convolutional_conv_weights
ModelImporter.cpp:129: 097_convolutional [Conv] inputs: [092_convolutional_lrelu -> (1, 256, 38, 38)], [097_convolutional_conv_weights -> (128, 256, 1, 1)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 256, 38, 38)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 128
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 38, 38)
ImporterContext.hpp:122: Registering layer: 097_convolutional for ONNX node: 097_convolutional
ImporterContext.hpp:97: Registering tensor: 097_convolutional for ONNX tensor: 097_convolutional
ModelImporter.cpp:180: 097_convolutional [Conv] outputs: [097_convolutional -> (1, 128, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 097_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 097_convolutional
ModelImporter.cpp:123: Searching for input: 097_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 097_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 097_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 097_convolutional_bn_var
ModelImporter.cpp:129: 097_convolutional_bn [BatchNormalization] inputs: [097_convolutional -> (1, 128, 38, 38)], [097_convolutional_bn_scale -> (128)], [097_convolutional_bn_bias -> (128)], [097_convolutional_bn_mean -> (128)], [097_convolutional_bn_var -> (128)], 
ImporterContext.hpp:122: Registering layer: 097_convolutional_bn for ONNX node: 097_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 097_convolutional_bn for ONNX tensor: 097_convolutional_bn
ModelImporter.cpp:180: 097_convolutional_bn [BatchNormalization] outputs: [097_convolutional_bn -> (1, 128, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 097_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 097_convolutional_bn
ModelImporter.cpp:129: 097_convolutional_lrelu [LeakyRelu] inputs: [097_convolutional_bn -> (1, 128, 38, 38)], 
ImporterContext.hpp:122: Registering layer: 097_convolutional_lrelu for ONNX node: 097_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 097_convolutional_lrelu for ONNX tensor: 097_convolutional_lrelu
ModelImporter.cpp:180: 097_convolutional_lrelu [LeakyRelu] outputs: [097_convolutional_lrelu -> (1, 128, 38, 38)], 
ModelImporter.cpp:107: Parsing node: 098_upsample [Resize]
ModelImporter.cpp:123: Searching for input: 097_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 098_upsample_roi
ModelImporter.cpp:123: Searching for input: 098_upsample_scale
ModelImporter.cpp:129: 098_upsample [Resize] inputs: [097_convolutional_lrelu -> (1, 128, 38, 38)], [098_upsample_roi -> (4)], [098_upsample_scale -> (4)], 
ImporterContext.hpp:122: Registering layer: 098_upsample for ONNX node: 098_upsample
ImporterContext.hpp:97: Registering tensor: 098_upsample for ONNX tensor: 098_upsample
ModelImporter.cpp:180: 098_upsample [Resize] outputs: [098_upsample -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 099_route [Concat]
ModelImporter.cpp:123: Searching for input: 098_upsample
ModelImporter.cpp:123: Searching for input: 037_shortcut
ModelImporter.cpp:129: 099_route [Concat] inputs: [098_upsample -> (1, 128, 76, 76)], [037_shortcut -> (1, 256, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 099_route for ONNX node: 099_route
ImporterContext.hpp:97: Registering tensor: 099_route for ONNX tensor: 099_route
ModelImporter.cpp:180: 099_route [Concat] outputs: [099_route -> (1, 384, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 100_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 099_route
ModelImporter.cpp:123: Searching for input: 100_convolutional_conv_weights
ModelImporter.cpp:129: 100_convolutional [Conv] inputs: [099_route -> (1, 384, 76, 76)], [100_convolutional_conv_weights -> (128, 384, 1, 1)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 384, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 128
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 76, 76)
ImporterContext.hpp:122: Registering layer: 100_convolutional for ONNX node: 100_convolutional
ImporterContext.hpp:97: Registering tensor: 100_convolutional for ONNX tensor: 100_convolutional
ModelImporter.cpp:180: 100_convolutional [Conv] outputs: [100_convolutional -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 100_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 100_convolutional
ModelImporter.cpp:123: Searching for input: 100_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 100_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 100_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 100_convolutional_bn_var
ModelImporter.cpp:129: 100_convolutional_bn [BatchNormalization] inputs: [100_convolutional -> (1, 128, 76, 76)], [100_convolutional_bn_scale -> (128)], [100_convolutional_bn_bias -> (128)], [100_convolutional_bn_mean -> (128)], [100_convolutional_bn_var -> (128)], 
ImporterContext.hpp:122: Registering layer: 100_convolutional_bn for ONNX node: 100_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 100_convolutional_bn for ONNX tensor: 100_convolutional_bn
ModelImporter.cpp:180: 100_convolutional_bn [BatchNormalization] outputs: [100_convolutional_bn -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 100_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 100_convolutional_bn
ModelImporter.cpp:129: 100_convolutional_lrelu [LeakyRelu] inputs: [100_convolutional_bn -> (1, 128, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 100_convolutional_lrelu for ONNX node: 100_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 100_convolutional_lrelu for ONNX tensor: 100_convolutional_lrelu
ModelImporter.cpp:180: 100_convolutional_lrelu [LeakyRelu] outputs: [100_convolutional_lrelu -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 101_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 100_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 101_convolutional_conv_weights
ModelImporter.cpp:129: 101_convolutional [Conv] inputs: [100_convolutional_lrelu -> (1, 128, 76, 76)], [101_convolutional_conv_weights -> (256, 128, 3, 3)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 128, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 256
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 256, 76, 76)
ImporterContext.hpp:122: Registering layer: 101_convolutional for ONNX node: 101_convolutional
ImporterContext.hpp:97: Registering tensor: 101_convolutional for ONNX tensor: 101_convolutional
ModelImporter.cpp:180: 101_convolutional [Conv] outputs: [101_convolutional -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 101_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 101_convolutional
ModelImporter.cpp:123: Searching for input: 101_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 101_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 101_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 101_convolutional_bn_var
ModelImporter.cpp:129: 101_convolutional_bn [BatchNormalization] inputs: [101_convolutional -> (1, 256, 76, 76)], [101_convolutional_bn_scale -> (256)], [101_convolutional_bn_bias -> (256)], [101_convolutional_bn_mean -> (256)], [101_convolutional_bn_var -> (256)], 
ImporterContext.hpp:122: Registering layer: 101_convolutional_bn for ONNX node: 101_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 101_convolutional_bn for ONNX tensor: 101_convolutional_bn
ModelImporter.cpp:180: 101_convolutional_bn [BatchNormalization] outputs: [101_convolutional_bn -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 101_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 101_convolutional_bn
ModelImporter.cpp:129: 101_convolutional_lrelu [LeakyRelu] inputs: [101_convolutional_bn -> (1, 256, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 101_convolutional_lrelu for ONNX node: 101_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 101_convolutional_lrelu for ONNX tensor: 101_convolutional_lrelu
ModelImporter.cpp:180: 101_convolutional_lrelu [LeakyRelu] outputs: [101_convolutional_lrelu -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 102_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 101_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 102_convolutional_conv_weights
ModelImporter.cpp:129: 102_convolutional [Conv] inputs: [101_convolutional_lrelu -> (1, 256, 76, 76)], [102_convolutional_conv_weights -> (128, 256, 1, 1)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 256, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 128
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 76, 76)
ImporterContext.hpp:122: Registering layer: 102_convolutional for ONNX node: 102_convolutional
ImporterContext.hpp:97: Registering tensor: 102_convolutional for ONNX tensor: 102_convolutional
ModelImporter.cpp:180: 102_convolutional [Conv] outputs: [102_convolutional -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 102_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 102_convolutional
ModelImporter.cpp:123: Searching for input: 102_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 102_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 102_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 102_convolutional_bn_var
ModelImporter.cpp:129: 102_convolutional_bn [BatchNormalization] inputs: [102_convolutional -> (1, 128, 76, 76)], [102_convolutional_bn_scale -> (128)], [102_convolutional_bn_bias -> (128)], [102_convolutional_bn_mean -> (128)], [102_convolutional_bn_var -> (128)], 
ImporterContext.hpp:122: Registering layer: 102_convolutional_bn for ONNX node: 102_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 102_convolutional_bn for ONNX tensor: 102_convolutional_bn
ModelImporter.cpp:180: 102_convolutional_bn [BatchNormalization] outputs: [102_convolutional_bn -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 102_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 102_convolutional_bn
ModelImporter.cpp:129: 102_convolutional_lrelu [LeakyRelu] inputs: [102_convolutional_bn -> (1, 128, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 102_convolutional_lrelu for ONNX node: 102_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 102_convolutional_lrelu for ONNX tensor: 102_convolutional_lrelu
ModelImporter.cpp:180: 102_convolutional_lrelu [LeakyRelu] outputs: [102_convolutional_lrelu -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 103_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 102_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 103_convolutional_conv_weights
ModelImporter.cpp:129: 103_convolutional [Conv] inputs: [102_convolutional_lrelu -> (1, 128, 76, 76)], [103_convolutional_conv_weights -> (256, 128, 3, 3)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 128, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 256
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 256, 76, 76)
ImporterContext.hpp:122: Registering layer: 103_convolutional for ONNX node: 103_convolutional
ImporterContext.hpp:97: Registering tensor: 103_convolutional for ONNX tensor: 103_convolutional
ModelImporter.cpp:180: 103_convolutional [Conv] outputs: [103_convolutional -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 103_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 103_convolutional
ModelImporter.cpp:123: Searching for input: 103_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 103_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 103_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 103_convolutional_bn_var
ModelImporter.cpp:129: 103_convolutional_bn [BatchNormalization] inputs: [103_convolutional -> (1, 256, 76, 76)], [103_convolutional_bn_scale -> (256)], [103_convolutional_bn_bias -> (256)], [103_convolutional_bn_mean -> (256)], [103_convolutional_bn_var -> (256)], 
ImporterContext.hpp:122: Registering layer: 103_convolutional_bn for ONNX node: 103_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 103_convolutional_bn for ONNX tensor: 103_convolutional_bn
ModelImporter.cpp:180: 103_convolutional_bn [BatchNormalization] outputs: [103_convolutional_bn -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 103_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 103_convolutional_bn
ModelImporter.cpp:129: 103_convolutional_lrelu [LeakyRelu] inputs: [103_convolutional_bn -> (1, 256, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 103_convolutional_lrelu for ONNX node: 103_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 103_convolutional_lrelu for ONNX tensor: 103_convolutional_lrelu
ModelImporter.cpp:180: 103_convolutional_lrelu [LeakyRelu] outputs: [103_convolutional_lrelu -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 104_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 103_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 104_convolutional_conv_weights
ModelImporter.cpp:129: 104_convolutional [Conv] inputs: [103_convolutional_lrelu -> (1, 256, 76, 76)], [104_convolutional_conv_weights -> (128, 256, 1, 1)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 256, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 128
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 76, 76)
ImporterContext.hpp:122: Registering layer: 104_convolutional for ONNX node: 104_convolutional
ImporterContext.hpp:97: Registering tensor: 104_convolutional for ONNX tensor: 104_convolutional
ModelImporter.cpp:180: 104_convolutional [Conv] outputs: [104_convolutional -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 104_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 104_convolutional
ModelImporter.cpp:123: Searching for input: 104_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 104_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 104_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 104_convolutional_bn_var
ModelImporter.cpp:129: 104_convolutional_bn [BatchNormalization] inputs: [104_convolutional -> (1, 128, 76, 76)], [104_convolutional_bn_scale -> (128)], [104_convolutional_bn_bias -> (128)], [104_convolutional_bn_mean -> (128)], [104_convolutional_bn_var -> (128)], 
ImporterContext.hpp:122: Registering layer: 104_convolutional_bn for ONNX node: 104_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 104_convolutional_bn for ONNX tensor: 104_convolutional_bn
ModelImporter.cpp:180: 104_convolutional_bn [BatchNormalization] outputs: [104_convolutional_bn -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 104_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 104_convolutional_bn
ModelImporter.cpp:129: 104_convolutional_lrelu [LeakyRelu] inputs: [104_convolutional_bn -> (1, 128, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 104_convolutional_lrelu for ONNX node: 104_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 104_convolutional_lrelu for ONNX tensor: 104_convolutional_lrelu
ModelImporter.cpp:180: 104_convolutional_lrelu [LeakyRelu] outputs: [104_convolutional_lrelu -> (1, 128, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 105_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 104_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 105_convolutional_conv_weights
ModelImporter.cpp:129: 105_convolutional [Conv] inputs: [104_convolutional_lrelu -> (1, 128, 76, 76)], [105_convolutional_conv_weights -> (256, 128, 3, 3)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 128, 76, 76)
Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 256
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 256, 76, 76)
ImporterContext.hpp:122: Registering layer: 105_convolutional for ONNX node: 105_convolutional
ImporterContext.hpp:97: Registering tensor: 105_convolutional for ONNX tensor: 105_convolutional
ModelImporter.cpp:180: 105_convolutional [Conv] outputs: [105_convolutional -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 105_convolutional_bn [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 105_convolutional
ModelImporter.cpp:123: Searching for input: 105_convolutional_bn_scale
ModelImporter.cpp:123: Searching for input: 105_convolutional_bn_bias
ModelImporter.cpp:123: Searching for input: 105_convolutional_bn_mean
ModelImporter.cpp:123: Searching for input: 105_convolutional_bn_var
ModelImporter.cpp:129: 105_convolutional_bn [BatchNormalization] inputs: [105_convolutional -> (1, 256, 76, 76)], [105_convolutional_bn_scale -> (256)], [105_convolutional_bn_bias -> (256)], [105_convolutional_bn_mean -> (256)], [105_convolutional_bn_var -> (256)], 
ImporterContext.hpp:122: Registering layer: 105_convolutional_bn for ONNX node: 105_convolutional_bn
ImporterContext.hpp:97: Registering tensor: 105_convolutional_bn for ONNX tensor: 105_convolutional_bn
ModelImporter.cpp:180: 105_convolutional_bn [BatchNormalization] outputs: [105_convolutional_bn -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 105_convolutional_lrelu [LeakyRelu]
ModelImporter.cpp:123: Searching for input: 105_convolutional_bn
ModelImporter.cpp:129: 105_convolutional_lrelu [LeakyRelu] inputs: [105_convolutional_bn -> (1, 256, 76, 76)], 
ImporterContext.hpp:122: Registering layer: 105_convolutional_lrelu for ONNX node: 105_convolutional_lrelu
ImporterContext.hpp:97: Registering tensor: 105_convolutional_lrelu for ONNX tensor: 105_convolutional_lrelu
ModelImporter.cpp:180: 105_convolutional_lrelu [LeakyRelu] outputs: [105_convolutional_lrelu -> (1, 256, 76, 76)], 
ModelImporter.cpp:107: Parsing node: 106_convolutional [Conv]
ModelImporter.cpp:123: Searching for input: 105_convolutional_lrelu
ModelImporter.cpp:123: Searching for input: 106_convolutional_conv_weights
ModelImporter.cpp:123: Searching for input: 106_convolutional_conv_bias
ModelImporter.cpp:129: 106_convolutional [Conv] inputs: [105_convolutional_lrelu -> (1, 256, 76, 76)], [106_convolutional_conv_weights -> (255, 256, 1, 1)], [106_convolutional_conv_bias -> (255)], 
builtin_op_importers.cpp:442: Convolution input dimensions: (1, 256, 76, 76)
builtin_op_importers.cpp:524: Using kernel: (1, 1), strides: (1, 1), padding: (0, 0), dilations: (1, 1), numOutputs: 255
builtin_op_importers.cpp:525: Convolution output dimensions: (1, 255, 76, 76)
ImporterContext.hpp:122: Registering layer: 106_convolutional for ONNX node: 106_convolutional
ImporterContext.hpp:97: Registering tensor: 106_convolutional_1 for ONNX tensor: 106_convolutional
ModelImporter.cpp:180: 106_convolutional [Conv] outputs: [106_convolutional -> (1, 255, 76, 76)], 
ModelImporter.cpp:494: Marking 082_convolutional_1 as output: 082_convolutional
ModelImporter.cpp:494: Marking 094_convolutional_1 as output: 094_convolutional
ModelImporter.cpp:494: Marking 106_convolutional_1 as output: 106_convolutional
 ----- Parsing of ONNX model /home/orcun/Main.Drive/src/perception/object_recognition/detection/tensorrt_yolo/data/yolov3.onnx is Done ---- 
Applying optimizations and building TRT CUDA engine...
Assertion failed: dims.d[i] >= 1
../builder/cudnnBuilderGraph.cpp:780
Aborting...

../builder/cudnnBuilderGraph.cpp (780) - Assertion Error in checkDimsSanity: 0 (dims.d[i] >= 1)

builder->buildEngineWithConfig does not log anyting even though it is using the same logger as networkParser and etc.

spolisetty · June 3, 2021, 5:19pm

Hi @orcdnz.

We are unable to reproduce this error on TensorRT version 8.0.0 EA. We recommend you to please use latest version.

&&&& PASSED TensorRT.trtexec [TensorRT v8000] # trtexec --onnx=yolov3.onnx --verbose
[06/03/2021-17:17:18] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1439, GPU 1366 (MiB)

Thank you.

orcdnz · June 7, 2021, 1:40pm

Version 7.2.1.6 solved the problem.