Using custom built model with ImageNet converted from h5 to onnx

Heya :)

I built my own keras model that out performs any transfer learning I attempted so I would like to deploy it using ImageNet on the Jetson.

I used keras2onnx to convert it. Just the default command line method, no custom code.

I have a feeling the error below may just be bad command syntax but I could be wrong so I have included an SVG of the model itself as well for inspection.

Thank you in advance for any advice!

command:

nvidia@nvidia-desktop:~/Desktop/Inference_Tester$ python3 imagenet.py --model=h5_test.onnx --labels=test_labels.txt --input_blob=input_0 --output_blob=output_0 "test_images/*.jpg" test_results/result_%i.jpg

output:

jetson.inference -- imageNet loading network using argv command line params

imageNet -- loading classification network model from:
         -- prototxt     (null)
         -- model        h5_test.onnx
         -- class_labels test_labels.txt
         -- input_blob   'input_0'
         -- output_blob  'output_0'
         -- batch_size   1

[TRT]    TensorRT version 7.1.3
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    detected model format - ONNX  (extension '.onnx')
[TRT]    desired precision specified for GPU: FASTEST
[TRT]    requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]    native precisions detected for GPU:  FP32, FP16
[TRT]    selecting fastest native precision for GPU:  FP16
[TRT]    attempting to open engine cache file h5_test.onnx.1.1.7103.GPU.FP16.engine
[TRT]    cache file not found, profiling network model on device GPU
[TRT]    device GPU, loading /usr/bin/ h5_test.onnx
----------------------------------------------------------------
Input filename:   h5_test.onnx
ONNX IR version:  0.0.7
Opset version:    13
Producer name:    keras2onnx
Producer version: 1.8.1
Domain:           onnxmltools
Model version:    0
Doc string:       
----------------------------------------------------------------
[TRT]    Plugin creator already registered - ::GridAnchor_TRT version 1
[TRT]    Plugin creator already registered - ::NMS_TRT version 1
[TRT]    Plugin creator already registered - ::Reorg_TRT version 1
[TRT]    Plugin creator already registered - ::Region_TRT version 1
[TRT]    Plugin creator already registered - ::Clip_TRT version 1
[TRT]    Plugin creator already registered - ::LReLU_TRT version 1
[TRT]    Plugin creator already registered - ::PriorBox_TRT version 1
[TRT]    Plugin creator already registered - ::Normalize_TRT version 1
[TRT]    Plugin creator already registered - ::RPROI_TRT version 1
[TRT]    Plugin creator already registered - ::BatchedNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Plugin creator already registered - ::CropAndResize version 1
[TRT]    Plugin creator already registered - ::DetectionLayer_TRT version 1
[TRT]    Plugin creator already registered - ::Proposal version 1
[TRT]    Plugin creator already registered - ::ProposalLayer_TRT version 1
[TRT]    Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[TRT]    Plugin creator already registered - ::ResizeNearest_TRT version 1
[TRT]    Plugin creator already registered - ::Split version 1
[TRT]    Plugin creator already registered - ::SpecialSlice_TRT version 1
[TRT]    Plugin creator already registered - ::InstanceNormalization_TRT version 1
[TRT]    ModelImporter.cpp:202: Adding network input: input_tensor_input with dtype: float32, dimensions: (-1, 96, 128, 3)
[TRT]    ImporterContext.hpp:116: Registering tensor: input_tensor_input for ONNX tensor: input_tensor_input
[TRT]    ModelImporter.cpp:90: Importing initializer: output_tensor/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: output_tensor/bias:0
[TRT]    ModelImporter.cpp:90: Importing initializer: fc2/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: fc2/bias:0
[TRT]    ModelImporter.cpp:90: Importing initializer: fc1/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: fc1/bias:0
[TRT]    ModelImporter.cpp:90: Importing initializer: scale
[TRT]    ModelImporter.cpp:90: Importing initializer: bias
[TRT]    ModelImporter.cpp:90: Importing initializer: mean
[TRT]    ModelImporter.cpp:90: Importing initializer: var
[TRT]    ModelImporter.cpp:90: Importing initializer: block1_conv2/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: block1_conv2/bias:0
[TRT]    ModelImporter.cpp:90: Importing initializer: scale1
[TRT]    ModelImporter.cpp:90: Importing initializer: bias1
[TRT]    ModelImporter.cpp:90: Importing initializer: mean1
[TRT]    ModelImporter.cpp:90: Importing initializer: var1
[TRT]    ModelImporter.cpp:90: Importing initializer: block1_conv1/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: block1_conv1/bias:0
[TRT]    ModelImporter.cpp:90: Importing initializer: scale2
[TRT]    ModelImporter.cpp:90: Importing initializer: bias2
[TRT]    ModelImporter.cpp:90: Importing initializer: mean2
[TRT]    ModelImporter.cpp:90: Importing initializer: var2
[TRT]    ModelImporter.cpp:90: Importing initializer: input_tensor/kernel:0
[TRT]    ModelImporter.cpp:90: Importing initializer: input_tensor/bias:0
[TRT]    ModelImporter.cpp:103: Parsing node: Transpose13 [Transpose]
[TRT]    ModelImporter.cpp:119: Searching for input: input_tensor_input
[TRT]    ModelImporter.cpp:125: Transpose13 [Transpose] inputs: [input_tensor_input -> (-1, 96, 128, 3)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Transpose13 for ONNX node: Transpose13
[TRT]    ImporterContext.hpp:116: Registering tensor: adjusted_input2 for ONNX tensor: adjusted_input2
[TRT]    ModelImporter.cpp:179: Transpose13 [Transpose] outputs: [adjusted_input2 -> (-1, 3, 96, 128)], 
[TRT]    ModelImporter.cpp:103: Parsing node: input_tensor [Conv]
[TRT]    ModelImporter.cpp:119: Searching for input: adjusted_input2
[TRT]    ModelImporter.cpp:119: Searching for input: input_tensor/kernel:0
[TRT]    ModelImporter.cpp:119: Searching for input: input_tensor/bias:0
[TRT]    ModelImporter.cpp:125: input_tensor [Conv] inputs: [adjusted_input2 -> (-1, 3, 96, 128)], [input_tensor/kernel:0 -> (64, 3, 3, 3)], [input_tensor/bias:0 -> (64)], 
[TRT]    builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 3, 96, 128)
[TRT]    ImporterContext.hpp:141: Registering layer: input_tensor for ONNX node: input_tensor
[TRT]    builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 64
[TRT]    builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 64, 96, 128)
[TRT]    ImporterContext.hpp:116: Registering tensor: convolution_output2 for ONNX tensor: convolution_output2
[TRT]    ModelImporter.cpp:179: input_tensor [Conv] outputs: [convolution_output2 -> (-1, 64, 96, 128)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Relu4 [Relu]
[TRT]    ModelImporter.cpp:119: Searching for input: convolution_output2
[TRT]    ModelImporter.cpp:125: Relu4 [Relu] inputs: [convolution_output2 -> (-1, 64, 96, 128)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Relu4 for ONNX node: Relu4
[TRT]    ImporterContext.hpp:116: Registering tensor: activation/Relu:0 for ONNX tensor: activation/Relu:0
[TRT]    ModelImporter.cpp:179: Relu4 [Relu] outputs: [activation/Relu:0 -> (-1, 64, 96, 128)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization [BatchNormalization]
[TRT]    ModelImporter.cpp:119: Searching for input: activation/Relu:0
[TRT]    ModelImporter.cpp:119: Searching for input: scale2
[TRT]    ModelImporter.cpp:119: Searching for input: bias2
[TRT]    ModelImporter.cpp:119: Searching for input: mean2
[TRT]    ModelImporter.cpp:119: Searching for input: var2
[TRT]    ModelImporter.cpp:125: batch_normalization [BatchNormalization] inputs: [activation/Relu:0 -> (-1, 64, 96, 128)], [scale2 -> (64)], [bias2 -> (64)], [mean2 -> (64)], [var2 -> (64)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization for ONNX node: batch_normalization
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_norm_output_buffer2 for ONNX tensor: batch_norm_output_buffer2
[TRT]    ModelImporter.cpp:179: batch_normalization [BatchNormalization] outputs: [batch_norm_output_buffer2 -> (-1, 64, 96, 128)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization/FusedBatchNormV3:0_pooling [MaxPool]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_norm_output_buffer2
[TRT]    ModelImporter.cpp:125: batch_normalization/FusedBatchNormV3:0_pooling [MaxPool] inputs: [batch_norm_output_buffer2 -> (-1, 64, 96, 128)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization/FusedBatchNormV3:0_pooling for ONNX node: batch_normalization/FusedBatchNormV3:0_pooling
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_normalization/FusedBatchNormV3:0_pooling0 for ONNX tensor: batch_normalization/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:179: batch_normalization/FusedBatchNormV3:0_pooling [MaxPool] outputs: [batch_normalization/FusedBatchNormV3:0_pooling0 -> (-1, 64, 48, 64)], 
[TRT]    ModelImporter.cpp:103: Parsing node: block1_conv1 [Conv]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_normalization/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:119: Searching for input: block1_conv1/kernel:0
[TRT]    ModelImporter.cpp:119: Searching for input: block1_conv1/bias:0
[TRT]    ModelImporter.cpp:125: block1_conv1 [Conv] inputs: [batch_normalization/FusedBatchNormV3:0_pooling0 -> (-1, 64, 48, 64)], [block1_conv1/kernel:0 -> (64, 64, 3, 3)], [block1_conv1/bias:0 -> (64)], 
[TRT]    builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 64, 48, 64)
[TRT]    ImporterContext.hpp:141: Registering layer: block1_conv1 for ONNX node: block1_conv1
[TRT]    builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 64
[TRT]    builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 64, 48, 64)
[TRT]    ImporterContext.hpp:116: Registering tensor: convolution_output1 for ONNX tensor: convolution_output1
[TRT]    ModelImporter.cpp:179: block1_conv1 [Conv] outputs: [convolution_output1 -> (-1, 64, 48, 64)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Relu3 [Relu]
[TRT]    ModelImporter.cpp:119: Searching for input: convolution_output1
[TRT]    ModelImporter.cpp:125: Relu3 [Relu] inputs: [convolution_output1 -> (-1, 64, 48, 64)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Relu3 for ONNX node: Relu3
[TRT]    ImporterContext.hpp:116: Registering tensor: activation_1/Relu:0 for ONNX tensor: activation_1/Relu:0
[TRT]    ModelImporter.cpp:179: Relu3 [Relu] outputs: [activation_1/Relu:0 -> (-1, 64, 48, 64)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization_1 [BatchNormalization]
[TRT]    ModelImporter.cpp:119: Searching for input: activation_1/Relu:0
[TRT]    ModelImporter.cpp:119: Searching for input: scale1
[TRT]    ModelImporter.cpp:119: Searching for input: bias1
[TRT]    ModelImporter.cpp:119: Searching for input: mean1
[TRT]    ModelImporter.cpp:119: Searching for input: var1
[TRT]    ModelImporter.cpp:125: batch_normalization_1 [BatchNormalization] inputs: [activation_1/Relu:0 -> (-1, 64, 48, 64)], [scale1 -> (64)], [bias1 -> (64)], [mean1 -> (64)], [var1 -> (64)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization_1 for ONNX node: batch_normalization_1
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_norm_output_buffer1 for ONNX tensor: batch_norm_output_buffer1
[TRT]    ModelImporter.cpp:179: batch_normalization_1 [BatchNormalization] outputs: [batch_norm_output_buffer1 -> (-1, 64, 48, 64)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization_1/FusedBatchNormV3:0_pooling [MaxPool]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_norm_output_buffer1
[TRT]    ModelImporter.cpp:125: batch_normalization_1/FusedBatchNormV3:0_pooling [MaxPool] inputs: [batch_norm_output_buffer1 -> (-1, 64, 48, 64)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization_1/FusedBatchNormV3:0_pooling for ONNX node: batch_normalization_1/FusedBatchNormV3:0_pooling
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_normalization_1/FusedBatchNormV3:0_pooling0 for ONNX tensor: batch_normalization_1/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:179: batch_normalization_1/FusedBatchNormV3:0_pooling [MaxPool] outputs: [batch_normalization_1/FusedBatchNormV3:0_pooling0 -> (-1, 64, 24, 32)], 
[TRT]    ModelImporter.cpp:103: Parsing node: block1_conv2 [Conv]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_normalization_1/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:119: Searching for input: block1_conv2/kernel:0
[TRT]    ModelImporter.cpp:119: Searching for input: block1_conv2/bias:0
[TRT]    ModelImporter.cpp:125: block1_conv2 [Conv] inputs: [batch_normalization_1/FusedBatchNormV3:0_pooling0 -> (-1, 64, 24, 32)], [block1_conv2/kernel:0 -> (64, 64, 3, 3)], [block1_conv2/bias:0 -> (64)], 
[TRT]    builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 64, 24, 32)
[TRT]    ImporterContext.hpp:141: Registering layer: block1_conv2 for ONNX node: block1_conv2
[TRT]    builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 64
[TRT]    builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 64, 24, 32)
[TRT]    ImporterContext.hpp:116: Registering tensor: convolution_output for ONNX tensor: convolution_output
[TRT]    ModelImporter.cpp:179: block1_conv2 [Conv] outputs: [convolution_output -> (-1, 64, 24, 32)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Relu2 [Relu]
[TRT]    ModelImporter.cpp:119: Searching for input: convolution_output
[TRT]    ModelImporter.cpp:125: Relu2 [Relu] inputs: [convolution_output -> (-1, 64, 24, 32)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Relu2 for ONNX node: Relu2
[TRT]    ImporterContext.hpp:116: Registering tensor: activation_2/Relu:0 for ONNX tensor: activation_2/Relu:0
[TRT]    ModelImporter.cpp:179: Relu2 [Relu] outputs: [activation_2/Relu:0 -> (-1, 64, 24, 32)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization_2 [BatchNormalization]
[TRT]    ModelImporter.cpp:119: Searching for input: activation_2/Relu:0
[TRT]    ModelImporter.cpp:119: Searching for input: scale
[TRT]    ModelImporter.cpp:119: Searching for input: bias
[TRT]    ModelImporter.cpp:119: Searching for input: mean
[TRT]    ModelImporter.cpp:119: Searching for input: var
[TRT]    ModelImporter.cpp:125: batch_normalization_2 [BatchNormalization] inputs: [activation_2/Relu:0 -> (-1, 64, 24, 32)], [scale -> (64)], [bias -> (64)], [mean -> (64)], [var -> (64)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization_2 for ONNX node: batch_normalization_2
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_norm_output_buffer for ONNX tensor: batch_norm_output_buffer
[TRT]    ModelImporter.cpp:179: batch_normalization_2 [BatchNormalization] outputs: [batch_norm_output_buffer -> (-1, 64, 24, 32)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization_2/FusedBatchNormV3:0_pooling [MaxPool]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_norm_output_buffer
[TRT]    ModelImporter.cpp:125: batch_normalization_2/FusedBatchNormV3:0_pooling [MaxPool] inputs: [batch_norm_output_buffer -> (-1, 64, 24, 32)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization_2/FusedBatchNormV3:0_pooling for ONNX node: batch_normalization_2/FusedBatchNormV3:0_pooling
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_normalization_2/FusedBatchNormV3:0_pooling0 for ONNX tensor: batch_normalization_2/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:179: batch_normalization_2/FusedBatchNormV3:0_pooling [MaxPool] outputs: [batch_normalization_2/FusedBatchNormV3:0_pooling0 -> (-1, 64, 12, 16)], 
[TRT]    ModelImporter.cpp:103: Parsing node: batch_normalization_2/FusedBatchNormV3:0_transpose [Transpose]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_normalization_2/FusedBatchNormV3:0_pooling0
[TRT]    ModelImporter.cpp:125: batch_normalization_2/FusedBatchNormV3:0_transpose [Transpose] inputs: [batch_normalization_2/FusedBatchNormV3:0_pooling0 -> (-1, 64, 12, 16)], 
[TRT]    ImporterContext.hpp:141: Registering layer: batch_normalization_2/FusedBatchNormV3:0_transpose for ONNX node: batch_normalization_2/FusedBatchNormV3:0_transpose
[TRT]    ImporterContext.hpp:116: Registering tensor: batch_normalization_2/FusedBatchNormV3:0_transpose0 for ONNX tensor: batch_normalization_2/FusedBatchNormV3:0_transpose0
[TRT]    ModelImporter.cpp:179: batch_normalization_2/FusedBatchNormV3:0_transpose [Transpose] outputs: [batch_normalization_2/FusedBatchNormV3:0_transpose0 -> (-1, 12, 16, 64)], 
[TRT]    ModelImporter.cpp:103: Parsing node: flatten [Flatten]
[TRT]    ModelImporter.cpp:119: Searching for input: batch_normalization_2/FusedBatchNormV3:0_transpose0
[TRT]    ModelImporter.cpp:125: flatten [Flatten] inputs: [batch_normalization_2/FusedBatchNormV3:0_transpose0 -> (-1, 12, 16, 64)], 
[TRT]    onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TRT]    ImporterContext.hpp:141: Registering layer: flatten for ONNX node: flatten
[TRT]    ImporterContext.hpp:116: Registering tensor: flatten/Reshape:0 for ONNX tensor: flatten/Reshape:0
[TRT]    ModelImporter.cpp:179: flatten [Flatten] outputs: [flatten/Reshape:0 -> (-1, -1)], 
[TRT]    ModelImporter.cpp:103: Parsing node: fc1 [MatMul]
[TRT]    ModelImporter.cpp:119: Searching for input: flatten/Reshape:0
[TRT]    ModelImporter.cpp:119: Searching for input: fc1/kernel:0
[TRT]    ModelImporter.cpp:125: fc1 [MatMul] inputs: [flatten/Reshape:0 -> (-1, -1)], [fc1/kernel:0 -> (12288, 256)], 
[TRT]    builtin_op_importers.cpp:2053: GEMM: using FC layer instead of MM because all criteria were met.
[TRT]    onnx2trt_utils.cpp:1793: Original shape: (_, _), unsqueezing to: (_, _, _, _)
[TRT]    ImporterContext.hpp:141: Registering layer: fc1 for ONNX node: fc1
[TRT]    onnx2trt_utils.cpp:1641: Original shape: (_, 256, 1, 1), squeezing to: (_, _)
[TRT]    ImporterContext.hpp:116: Registering tensor: fc10 for ONNX tensor: fc10
[TRT]    ModelImporter.cpp:179: fc1 [MatMul] outputs: [fc10 -> (-1, -1)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Add2 [Add]
[TRT]    ModelImporter.cpp:119: Searching for input: fc10
[TRT]    ModelImporter.cpp:119: Searching for input: fc1/bias:0
[TRT]    ModelImporter.cpp:125: Add2 [Add] inputs: [fc10 -> (-1, -1)], [fc1/bias:0 -> (256)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Add2 for ONNX node: Add2
[TRT]    ImporterContext.hpp:116: Registering tensor: biased_tensor_name2 for ONNX tensor: biased_tensor_name2
[TRT]    ModelImporter.cpp:179: Add2 [Add] outputs: [biased_tensor_name2 -> (-1, 256)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Relu1 [Relu]
[TRT]    ModelImporter.cpp:119: Searching for input: biased_tensor_name2
[TRT]    ModelImporter.cpp:125: Relu1 [Relu] inputs: [biased_tensor_name2 -> (-1, 256)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Relu1 for ONNX node: Relu1
[TRT]    ImporterContext.hpp:116: Registering tensor: fc1/Relu:0 for ONNX tensor: fc1/Relu:0
[TRT]    ModelImporter.cpp:179: Relu1 [Relu] outputs: [fc1/Relu:0 -> (-1, 256)], 
[TRT]    ModelImporter.cpp:103: Parsing node: fc2 [MatMul]
[TRT]    ModelImporter.cpp:119: Searching for input: fc1/Relu:0
[TRT]    ModelImporter.cpp:119: Searching for input: fc2/kernel:0
[TRT]    ModelImporter.cpp:125: fc2 [MatMul] inputs: [fc1/Relu:0 -> (-1, 256)], [fc2/kernel:0 -> (256, 256)], 
[TRT]    builtin_op_importers.cpp:2053: GEMM: using FC layer instead of MM because all criteria were met.
[TRT]    onnx2trt_utils.cpp:1793: Original shape: (_, 256), unsqueezing to: (_, _, _, _)
[TRT]    ImporterContext.hpp:141: Registering layer: fc2 for ONNX node: fc2
[TRT]    onnx2trt_utils.cpp:1641: Original shape: (_, 256, 1, 1), squeezing to: (_, _)
[TRT]    ImporterContext.hpp:116: Registering tensor: fc20 for ONNX tensor: fc20
[TRT]    ModelImporter.cpp:179: fc2 [MatMul] outputs: [fc20 -> (-1, -1)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Add1 [Add]
[TRT]    ModelImporter.cpp:119: Searching for input: fc20
[TRT]    ModelImporter.cpp:119: Searching for input: fc2/bias:0
[TRT]    ModelImporter.cpp:125: Add1 [Add] inputs: [fc20 -> (-1, -1)], [fc2/bias:0 -> (256)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Add1 for ONNX node: Add1
[TRT]    ImporterContext.hpp:116: Registering tensor: biased_tensor_name1 for ONNX tensor: biased_tensor_name1
[TRT]    ModelImporter.cpp:179: Add1 [Add] outputs: [biased_tensor_name1 -> (-1, 256)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Relu [Relu]
[TRT]    ModelImporter.cpp:119: Searching for input: biased_tensor_name1
[TRT]    ModelImporter.cpp:125: Relu [Relu] inputs: [biased_tensor_name1 -> (-1, 256)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Relu for ONNX node: Relu
[TRT]    ImporterContext.hpp:116: Registering tensor: fc2/Relu:0 for ONNX tensor: fc2/Relu:0
[TRT]    ModelImporter.cpp:179: Relu [Relu] outputs: [fc2/Relu:0 -> (-1, 256)], 
[TRT]    ModelImporter.cpp:103: Parsing node: output_tensor [MatMul]
[TRT]    ModelImporter.cpp:119: Searching for input: fc2/Relu:0
[TRT]    ModelImporter.cpp:119: Searching for input: output_tensor/kernel:0
[TRT]    ModelImporter.cpp:125: output_tensor [MatMul] inputs: [fc2/Relu:0 -> (-1, 256)], [output_tensor/kernel:0 -> (256, 2)], 
[TRT]    builtin_op_importers.cpp:2053: GEMM: using FC layer instead of MM because all criteria were met.
[TRT]    onnx2trt_utils.cpp:1793: Original shape: (_, 256), unsqueezing to: (_, _, _, _)
[TRT]    ImporterContext.hpp:141: Registering layer: output_tensor for ONNX node: output_tensor
[TRT]    onnx2trt_utils.cpp:1641: Original shape: (_, 2, 1, 1), squeezing to: (_, _)
[TRT]    ImporterContext.hpp:116: Registering tensor: output_tensor0 for ONNX tensor: output_tensor0
[TRT]    ModelImporter.cpp:179: output_tensor [MatMul] outputs: [output_tensor0 -> (-1, -1)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Add [Add]
[TRT]    ModelImporter.cpp:119: Searching for input: output_tensor0
[TRT]    ModelImporter.cpp:119: Searching for input: output_tensor/bias:0
[TRT]    ModelImporter.cpp:125: Add [Add] inputs: [output_tensor0 -> (-1, -1)], [output_tensor/bias:0 -> (2)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Add for ONNX node: Add
[TRT]    ImporterContext.hpp:116: Registering tensor: biased_tensor_name for ONNX tensor: biased_tensor_name
[TRT]    ModelImporter.cpp:179: Add [Add] outputs: [biased_tensor_name -> (-1, 2)], 
[TRT]    ModelImporter.cpp:103: Parsing node: Softmax [Softmax]
[TRT]    ModelImporter.cpp:119: Searching for input: biased_tensor_name
[TRT]    ModelImporter.cpp:125: Softmax [Softmax] inputs: [biased_tensor_name -> (-1, 2)], 
[TRT]    ImporterContext.hpp:141: Registering layer: Softmax for ONNX node: Softmax
[TRT]    ImporterContext.hpp:116: Registering tensor: output_tensor_1 for ONNX tensor: output_tensor
[TRT]    ModelImporter.cpp:179: Softmax [Softmax] outputs: [output_tensor -> (-1, -1)], 
[TRT]    ModelImporter.cpp:507: Marking output_tensor_1 as output: output_tensor
 ----- Parsing of ONNX model h5_test.onnx is Done ---- 
[TRT]    device GPU, configuring network builder
[TRT]    device GPU, building FP16:  ON
[TRT]    device GPU, building INT8:  OFF
[TRT]    device GPU, workspace size: 33554432
[TRT]    device GPU, building CUDA engine (this may take a few minutes the first time a network is loaded)
[TRT]    Network has dynamic or shape inputs, but no optimization profile has been defined.
[TRT]    Network validation failed.
[TRT]    device GPU, failed to build CUDA engine
[TRT]    device GPU, failed to load h5_test.onnx
[TRT]    failed to load h5_test.onnx
[TRT]    imageNet -- failed to initialize.
jetson.inference -- imageNet failed to load built-in network 'googlenet'
Traceback (most recent call last):
  File "imagenet.py", line 55, in <module>
    net = jetson.inference.imageNet(opt.network, sys.argv)
Exception: jetson.inference -- imageNet failed to load network

keras model:
h5_test.h5

onnx converted:
h5_test.onnx

Hi,

[TRT] Network has dynamic or shape inputs, but no optimization profile has been defined.

This is a TensorRT error about the usage of the dynamic input model.
Would you mind reconverting your model into a fixed input size ONNX model and try it again?

Thanks.

1 Like

Thanks I got it working using below code if anyone stumbles on this in future:

import onnx


def change_input_dim(model):
    # Use some symbolic name not used for any other dimension
    sym_batch_dim = "N"
    # or an actal value
    actual_batch_dim = 1

    # The following code changes the first dimension of every input to be batch-dim
    # Modify as appropriate ... note that this requires all inputs to
    # have the same batch_dim
    inputs = model.graph.input
    for input in inputs:
        # Checks omitted.This assumes that all inputs are tensors and have a shape with first dim.
        # Add checks as needed.
        dim1 = input.type.tensor_type.shape.dim[0]
        # update dim to be a symbolic value
        # dim1.dim_param = sym_batch_dim
        # or update it to be an actual value:
        dim1.dim_value = actual_batch_dim


def apply(transform, infile, outfile):
    model = onnx.load(infile)
    transform(model)
    onnx.save(model, outfile)


apply(change_input_dim, r"h5_test.onnx", r"new_h5.onnx")

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.