Yolor to onnx to trt

Description

I’ve trained a yolor model (GitHub - WongKinYiu/yolor: implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks (https://arxiv.org/abs/2105.04206)) and I’ve exported it to onnx this snippet of code.

git clone https://github.com/WongKinYiu/yolor.git

then

import sys 

sys.path.append('yolor')

from yolor.models.models import Darknet
import numpy as np
import torch
import onnx

device = 'cuda'
half = False
weight_file = 'yolor_csp.pt'
img_size = 640
img = torch.rand((1, 3,img_size, img_size)).to(device)

model = Darknet('yolor/cfg/yolor_csp.cfg', img_size).cuda()

print('\nStarting ONNX export with onnx %s...' % onnx.__version__)
f = weight_file.replace('.pt', '.onnx')  # filename
model.fuse()  # only for ONNX
torch.onnx.export(model, img, f, verbose=False, opset_version=12, input_names=['images'],
                  output_names=['classes', 'boxes'] if y is None else ['output'])

onnx_model = onnx.load(f)  # load onnx model
onnx.checker.check_model(onnx_model)  # check onnx model
print(onnx.helper.printable_graph(onnx_model.graph))  # print a human readable model
print('ONNX export success, saved as %s' % f)

the model I’m using is Darknet model (yolor/models.py at main · WongKinYiu/yolor · GitHub)

I tried to run polygraph to check if the results are accurate and this is the output of the command

[I] Will generate inference input data according to provided TensorMetadata: {images [shape=(1, 3, 640, 640)]}
[I] trt-runner-N0-09/14/22-11:08:17     | Activating and starting inference
[09/14/2022-11:08:21] [TRT] [W] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[09/14/2022-11:08:21] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:21] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:22] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:23] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:23] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:23] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[09/14/2022-11:08:23] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[I]     Configuring with profiles: [Profile().add('images', min=[1, 3, 640, 640], opt=[1, 3, 640, 640], max=[1, 3, 640, 640])]
[I] Building engine with configuration:
    Workspace            | 16777216 bytes (16.00 MiB)
    Precision            | TF32: False, FP16: False, INT8: False, Obey Precision Constraints: False, Strict Types: False
    Tactic Sources       | ['CUBLAS', 'CUBLAS_LT', 'CUDNN', 'EDGE_MASK_CONVOLUTIONS']
    Safety Restricted    | False
    Profiles             | 1 profile(s)
[09/14/2022-11:11:13] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[09/14/2022-11:11:13] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[I] Finished engine building in 169.685 seconds
[I] trt-runner-N0-09/14/22-11:08:17    
    ---- Inference Input(s) ----
    {images [dtype=float32, shape=(1, 3, 640, 640)]}
[I] trt-runner-N0-09/14/22-11:08:17    
    ---- Inference Output(s) ----
    {onnx::Sigmoid_638 [dtype=float32, shape=(1, 3, 80, 80, 7)],
     onnx::Sigmoid_977 [dtype=float32, shape=(1, 3, 40, 40, 7)],
     onnx::Sigmoid_1316 [dtype=float32, shape=(1, 3, 20, 20, 7)],
     output [dtype=float32, shape=(1, 25200, 7)]}
[I] trt-runner-N0-09/14/22-11:08:17     | Completed 1 iteration(s) in 67.43 ms | Average inference time: 67.43 ms.
[I] onnxrt-runner-N0-09/14/22-11:08:17  | Activating and starting inference
[I] Creating ONNX-Runtime Inference Session with providers: ['CPUExecutionProvider']
[I] onnxrt-runner-N0-09/14/22-11:08:17 
    ---- Inference Input(s) ----
    {images [dtype=float32, shape=(1, 3, 640, 640)]}
[I] onnxrt-runner-N0-09/14/22-11:08:17 
    ---- Inference Output(s) ----
    {output [dtype=float32, shape=(1, 25200, 7)],
     onnx::Sigmoid_638 [dtype=float32, shape=(1, 3, 80, 80, 7)],
     onnx::Sigmoid_977 [dtype=float32, shape=(1, 3, 40, 40, 7)],
     onnx::Sigmoid_1316 [dtype=float32, shape=(1, 3, 20, 20, 7)]}
[I] onnxrt-runner-N0-09/14/22-11:08:17  | Completed 1 iteration(s) in 801.1 ms | Average inference time: 801.1 ms.
[I] Accuracy Comparison | trt-runner-N0-09/14/22-11:08:17 vs. onnxrt-runner-N0-09/14/22-11:08:17
[I]     Comparing Output: 'onnx::Sigmoid_638' (dtype=float32, shape=(1, 3, 80, 80, 7)) with 'onnx::Sigmoid_638' (dtype=float32, shape=(1, 3, 80, 80, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_638 | Stats: mean=-1.5821, std-dev=4.023, var=16.185, median=-0.035941, min=-13.831 at (0, 0, 69, 1, 4), max=2.5998 at (0, 0, 65, 56, 5), avg-magnitude=1.9761
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.8 , -12.2 ) |       3189 | #
                (-12.2 , -10.5 ) |      12571 | #####
                (-10.5 , -8.9  ) |       3174 | #
                (-8.9  , -7.26 ) |        265 | 
                (-7.26 , -5.62 ) |          1 | 
                (-5.62 , -3.97 ) |          0 | 
                (-3.97 , -2.33 ) |         14 | 
                (-2.33 , -0.686) |       9883 | ###
                (-0.686, 0.957 ) |      99604 | ########################################
                (0.957 , 2.6   ) |       5699 | ##
[I]         onnxrt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_638 | Stats: mean=-1.5821, std-dev=4.023, var=16.185, median=-0.035944, min=-13.831 at (0, 0, 69, 1, 4), max=2.5998 at (0, 0, 65, 56, 5), avg-magnitude=1.9761
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.8 , -12.2 ) |       3189 | #
                (-12.2 , -10.5 ) |      12571 | #####
                (-10.5 , -8.9  ) |       3174 | #
                (-8.9  , -7.26 ) |        265 | 
                (-7.26 , -5.62 ) |          1 | 
                (-5.62 , -3.97 ) |          0 | 
                (-3.97 , -2.33 ) |         14 | 
                (-2.33 , -0.686) |       9883 | ###
                (-0.686, 0.957 ) |      99604 | ########################################
                (0.957 , 2.6   ) |       5699 | ##
[I]         Error Metrics: onnx::Sigmoid_638
[I]             Minimum Required Tolerance: elemwise error | [abs=7.1526e-05] OR [rel=1.4975] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=5.3967e-06, std-dev=6.0205e-06, var=3.6247e-11, median=3.5465e-06, min=0 at (0, 0, 0, 4, 5), max=7.1526e-05 at (0, 1, 44, 42, 4), avg-magnitude=5.3967e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 7.15e-06) |     101980 | ########################################
                    (7.15e-06, 1.43e-05) |      21997 | ########
                    (1.43e-05, 2.15e-05) |       6389 | ##
                    (2.15e-05, 2.86e-05) |       2356 | 
                    (2.86e-05, 3.58e-05) |       1149 | 
                    (3.58e-05, 4.29e-05) |        307 | 
                    (4.29e-05, 5.01e-05) |        154 | 
                    (5.01e-05, 5.72e-05) |         40 | 
                    (5.72e-05, 6.44e-05) |         20 | 
                    (6.44e-05, 7.15e-05) |          8 | 
[I]             Relative Difference | Stats: mean=7.0739e-05, std-dev=0.0043159, var=1.8627e-05, median=7.4942e-06, min=0 at (0, 0, 0, 4, 5), max=1.4975 at (0, 1, 12, 7, 0), avg-magnitude=7.0739e-05
[I]                 ---- Histogram ----
                    Bin Range      |  Num Elems | Visualization
                    (0    , 0.15 ) |     134396 | ########################################
                    (0.15 , 0.3  ) |          3 | 
                    (0.3  , 0.449) |          0 | 
                    (0.449, 0.599) |          0 | 
                    (0.599, 0.749) |          0 | 
                    (0.749, 0.899) |          0 | 
                    (0.899, 1.05 ) |          0 | 
                    (1.05 , 1.2  ) |          0 | 
                    (1.2  , 1.35 ) |          0 | 
                    (1.35 , 1.5  ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'onnx::Sigmoid_977' (dtype=float32, shape=(1, 3, 40, 40, 7)) with 'onnx::Sigmoid_977' (dtype=float32, shape=(1, 3, 40, 40, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_977 | Stats: mean=-1.3664, std-dev=3.6837, var=13.57, median=-0.054346, min=-13.749 at (0, 2, 39, 12, 4), max=2.6531 at (0, 2, 26, 5, 5), avg-magnitude=1.9664
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.7 , -12.1 ) |         45 | 
                (-12.1 , -10.5 ) |       1769 | ###
                (-10.5 , -8.83 ) |       2720 | #####
                (-8.83 , -7.19 ) |        266 | 
                (-7.19 , -5.55 ) |          0 | 
                (-5.55 , -3.91 ) |          0 | 
                (-3.91 , -2.27 ) |        103 | 
                (-2.27 , -0.627) |       4179 | ########
                (-0.627, 1.01  ) |      20353 | ########################################
                (1.01  , 2.65  ) |       4165 | ########
[I]         onnxrt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_977 | Stats: mean=-1.3664, std-dev=3.6837, var=13.57, median=-0.054349, min=-13.749 at (0, 2, 39, 12, 4), max=2.6531 at (0, 2, 26, 5, 5), avg-magnitude=1.9664
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.7 , -12.1 ) |         45 | 
                (-12.1 , -10.5 ) |       1769 | ###
                (-10.5 , -8.83 ) |       2720 | #####
                (-8.83 , -7.19 ) |        266 | 
                (-7.19 , -5.55 ) |          0 | 
                (-5.55 , -3.91 ) |          0 | 
                (-3.91 , -2.27 ) |        103 | 
                (-2.27 , -0.627) |       4179 | ########
                (-0.627, 1.01  ) |      20353 | ########################################
                (1.01  , 2.65  ) |       4165 | ########
[I]         Error Metrics: onnx::Sigmoid_977
[I]             Minimum Required Tolerance: elemwise error | [abs=4.4823e-05] OR [rel=0.52614] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.733e-06, std-dev=4.1101e-06, var=1.6893e-11, median=2.4438e-06, min=0 at (0, 0, 0, 13, 4), max=4.4823e-05 at (0, 0, 21, 11, 4), avg-magnitude=3.733e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 4.48e-06) |      24109 | ########################################
                    (4.48e-06, 8.96e-06) |       6489 | ##########
                    (8.96e-06, 1.34e-05) |       1825 | ###
                    (1.34e-05, 1.79e-05) |        609 | #
                    (1.79e-05, 2.24e-05) |        354 | 
                    (2.24e-05, 2.69e-05) |        134 | 
                    (2.69e-05, 3.14e-05) |         50 | 
                    (3.14e-05, 3.59e-05) |         23 | 
                    (3.59e-05, 4.03e-05) |          5 | 
                    (4.03e-05, 4.48e-05) |          2 | 
[I]             Relative Difference | Stats: mean=5.4347e-05, std-dev=0.0029416, var=8.6528e-06, median=3.9592e-06, min=0 at (0, 0, 0, 13, 4), max=0.52614 at (0, 2, 29, 18, 0), avg-magnitude=5.4347e-05
[I]                 ---- Histogram ----
                    Bin Range        |  Num Elems | Visualization
                    (0     , 0.0526) |      33598 | ########################################
                    (0.0526, 0.105 ) |          1 | 
                    (0.105 , 0.158 ) |          0 | 
                    (0.158 , 0.21  ) |          0 | 
                    (0.21  , 0.263 ) |          0 | 
                    (0.263 , 0.316 ) |          0 | 
                    (0.316 , 0.368 ) |          0 | 
                    (0.368 , 0.421 ) |          0 | 
                    (0.421 , 0.474 ) |          0 | 
                    (0.474 , 0.526 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'onnx::Sigmoid_1316' (dtype=float32, shape=(1, 3, 20, 20, 7)) with 'onnx::Sigmoid_1316' (dtype=float32, shape=(1, 3, 20, 20, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_1316 | Stats: mean=-1.261, std-dev=3.3875, var=11.475, median=-0.12893, min=-14.461 at (0, 2, 19, 0, 4), max=3.371 at (0, 2, 12, 3, 5), avg-magnitude=1.9811
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-14.5 , -12.7 ) |         32 | 
                (-12.7 , -10.9 ) |         72 | 
                (-10.9 , -9.11 ) |        480 | #####
                (-9.11 , -7.33 ) |        496 | #####
                (-7.33 , -5.55 ) |        120 | #
                (-5.55 , -3.76 ) |          0 | 
                (-3.76 , -1.98 ) |        253 | ##
                (-1.98 , -0.195) |       2468 | ##########################
                (-0.195, 1.59  ) |       3738 | ########################################
                (1.59  , 3.37  ) |        741 | #######
[I]         onnxrt-runner-N0-09/14/22-11:08:17: onnx::Sigmoid_1316 | Stats: mean=-1.261, std-dev=3.3875, var=11.475, median=-0.12893, min=-14.461 at (0, 2, 19, 0, 4), max=3.371 at (0, 2, 12, 3, 5), avg-magnitude=1.9811
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-14.5 , -12.7 ) |         32 | 
                (-12.7 , -10.9 ) |         72 | 
                (-10.9 , -9.11 ) |        480 | #####
                (-9.11 , -7.33 ) |        496 | #####
                (-7.33 , -5.55 ) |        120 | #
                (-5.55 , -3.76 ) |          0 | 
                (-3.76 , -1.98 ) |        253 | ##
                (-1.98 , -0.195) |       2468 | ##########################
                (-0.195, 1.59  ) |       3738 | ########################################
                (1.59  , 3.37  ) |        741 | #######
[I]         Error Metrics: onnx::Sigmoid_1316
[I]             Minimum Required Tolerance: elemwise error | [abs=3.767e-05] OR [rel=0.11636] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.1095e-06, std-dev=3.6627e-06, var=1.3415e-11, median=1.8603e-06, min=0 at (0, 0, 1, 7, 4), max=3.767e-05 at (0, 1, 9, 2, 4), avg-magnitude=3.1095e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 3.77e-06) |       6009 | ########################################
                    (3.77e-06, 7.53e-06) |       1501 | #########
                    (7.53e-06, 1.13e-05) |        525 | ###
                    (1.13e-05, 1.51e-05) |        211 | #
                    (1.51e-05, 1.88e-05) |         87 | 
                    (1.88e-05, 2.26e-05) |         48 | 
                    (2.26e-05, 2.64e-05) |         13 | 
                    (2.64e-05, 3.01e-05) |          2 | 
                    (3.01e-05, 3.39e-05) |          1 | 
                    (3.39e-05, 3.77e-05) |          3 | 
[I]             Relative Difference | Stats: mean=4.7262e-05, std-dev=0.0015394, var=2.3698e-06, median=2.2524e-06, min=0 at (0, 0, 1, 7, 4), max=0.11636 at (0, 0, 7, 16, 1), avg-magnitude=4.7262e-05
[I]                 ---- Histogram ----
                    Bin Range        |  Num Elems | Visualization
                    (0     , 0.0116) |       8397 | ########################################
                    (0.0116, 0.0233) |          1 | 
                    (0.0233, 0.0349) |          0 | 
                    (0.0349, 0.0465) |          0 | 
                    (0.0465, 0.0582) |          0 | 
                    (0.0582, 0.0698) |          0 | 
                    (0.0698, 0.0814) |          1 | 
                    (0.0814, 0.0931) |          0 | 
                    (0.0931, 0.105 ) |          0 | 
                    (0.105 , 0.116 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'output' (dtype=float32, shape=(1, 25200, 7)) with 'output' (dtype=float32, shape=(1, 25200, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-11:08:17: output | Stats: mean=106.78, std-dev=173.18, var=29992, median=18.356, min=5.2426e-07 at (0, 25180, 4), max=637.66 at (0, 2639, 0), avg-magnitude=106.78
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (5.24e-07, 63.8) |     122026 | ########################################
                (63.8    , 128 ) |       9459 | ###
                (128     , 191 ) |       7071 | ##
                (191     , 255 ) |       5742 | #
                (255     , 319 ) |       5580 | #
                (319     , 383 ) |       5685 | #
                (383     , 446 ) |       5435 | #
                (446     , 510 ) |       5163 | #
                (510     , 574 ) |       5120 | #
                (574     , 638 ) |       5119 | #
[I]         onnxrt-runner-N0-09/14/22-11:08:17: output | Stats: mean=106.78, std-dev=173.18, var=29992, median=18.356, min=5.3644e-07 at (0, 25180, 4), max=637.66 at (0, 2639, 0), avg-magnitude=106.78
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (5.24e-07, 63.8) |     122026 | ########################################
                (63.8    , 128 ) |       9459 | ###
                (128     , 191 ) |       7071 | ##
                (191     , 255 ) |       5742 | #
                (255     , 319 ) |       5580 | #
                (319     , 383 ) |       5685 | #
                (383     , 446 ) |       5435 | #
                (446     , 510 ) |       5163 | #
                (510     , 574 ) |       5120 | #
                (574     , 638 ) |       5119 | #
[I]         Error Metrics: output
[I]             Minimum Required Tolerance: elemwise error | [abs=0.002594] OR [rel=0.077808] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.5768e-05, std-dev=9.4047e-05, var=8.8448e-09, median=1.5497e-06, min=0 at (0, 4, 5), max=0.002594 at (0, 23419, 3), avg-magnitude=3.5768e-05
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 0.000259) |     171100 | ########################################
                    (0.000259, 0.000519) |       3908 | 
                    (0.000519, 0.000778) |        960 | 
                    (0.000778, 0.00104 ) |        266 | 
                    (0.00104 , 0.0013  ) |         94 | 
                    (0.0013  , 0.00156 ) |         38 | 
                    (0.00156 , 0.00182 ) |         23 | 
                    (0.00182 , 0.00208 ) |          5 | 
                    (0.00208 , 0.00233 ) |          4 | 
                    (0.00233 , 0.00259 ) |          2 | 
[I]             Relative Difference | Stats: mean=0.00037051, std-dev=0.0016869, var=2.8458e-06, median=1.2529e-06, min=0 at (0, 4, 5), max=0.077808 at (0, 24800, 4), avg-magnitude=0.00037051
[I]                 ---- Histogram ----
                    Bin Range          |  Num Elems | Visualization
                    (0      , 0.00778) |     174425 | ########################################
                    (0.00778, 0.0156 ) |       1618 | 
                    (0.0156 , 0.0233 ) |        267 | 
                    (0.0233 , 0.0311 ) |         61 | 
                    (0.0311 , 0.0389 ) |         16 | 
                    (0.0389 , 0.0467 ) |          6 | 
                    (0.0467 , 0.0545 ) |          2 | 
                    (0.0545 , 0.0622 ) |          3 | 
                    (0.0622 , 0.07   ) |          1 | 
                    (0.07   , 0.0778 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[E]     FAILED | Mismatched outputs: ['onnx::Sigmoid_638', 'onnx::Sigmoid_977', 'onnx::Sigmoid_1316', 'output']
[!] FAILED | Command: /usr/local/bin/polygraphy run best_units.onnx --trt --onnxrt --input-shapes images:[1,3,640,640]

As you can see the network returns “FAILED”, there are some warnings

Environment

Device: Jetson AGX Orin Dev Kit
Jetpack: 5.0.2
Baremetal or Container (if container which image + tag): Container nvidia/l4t-ml:r35.1.0-py3

Relevant Files

yolor_csp.pt - Google Drive the weights of the net in .pt format

Issue Reproduction

Convert the .pt to onnx using the above script

run poligraphy to check output

polygraphy run yolo_csp.onnx --trt --onnxrt

Further testing

I tried to sanitize my model using this:

polygraphy surgeon sanitize yolor_csp.onnx --fold-constants -o folded.onnx

The result is of the polygraphy run is very similar:

[I] trt-runner-N0-09/14/22-13:32:43     | Activating and starting inference
[09/14/2022-13:32:47] [TRT] [W] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[I]     Configuring with profiles: [Profile().add('images', min=[1, 3, 640, 640], opt=[1, 3, 640, 640], max=[1, 3, 640, 640])]
[I] Building engine with configuration:
    Workspace            | 16777216 bytes (16.00 MiB)
    Precision            | TF32: False, FP16: False, INT8: False, Obey Precision Constraints: False, Strict Types: False
    Tactic Sources       | ['CUBLAS', 'CUBLAS_LT', 'CUDNN', 'EDGE_MASK_CONVOLUTIONS']
    Safety Restricted    | False
    Profiles             | 1 profile(s)
[09/14/2022-13:34:47] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[09/14/2022-13:34:47] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[I] Finished engine building in 120.563 seconds
[I] trt-runner-N0-09/14/22-13:32:43    
    ---- Inference Input(s) ----
    {images [dtype=float32, shape=(1, 3, 640, 640)]}
[I] trt-runner-N0-09/14/22-13:32:43    
    ---- Inference Output(s) ----
    {onnx::Sigmoid_638 [dtype=float32, shape=(1, 3, 80, 80, 7)],
     onnx::Sigmoid_977 [dtype=float32, shape=(1, 3, 40, 40, 7)],
     onnx::Sigmoid_1316 [dtype=float32, shape=(1, 3, 20, 20, 7)],
     output [dtype=float32, shape=(1, 25200, 7)]}
[I] trt-runner-N0-09/14/22-13:32:43     | Completed 1 iteration(s) in 50.5 ms | Average inference time: 50.5 ms.
[I] onnxrt-runner-N0-09/14/22-13:32:43  | Activating and starting inference
[I] Creating ONNX-Runtime Inference Session with providers: ['CPUExecutionProvider']
[I] onnxrt-runner-N0-09/14/22-13:32:43 
    ---- Inference Input(s) ----
    {images [dtype=float32, shape=(1, 3, 640, 640)]}
[I] onnxrt-runner-N0-09/14/22-13:32:43 
    ---- Inference Output(s) ----
    {output [dtype=float32, shape=(1, 25200, 7)],
     onnx::Sigmoid_638 [dtype=float32, shape=(1, 3, 80, 80, 7)],
     onnx::Sigmoid_977 [dtype=float32, shape=(1, 3, 40, 40, 7)],
     onnx::Sigmoid_1316 [dtype=float32, shape=(1, 3, 20, 20, 7)]}
[I] onnxrt-runner-N0-09/14/22-13:32:43  | Completed 1 iteration(s) in 784.9 ms | Average inference time: 784.9 ms.
[I] Accuracy Comparison | trt-runner-N0-09/14/22-13:32:43 vs. onnxrt-runner-N0-09/14/22-13:32:43
[I]     Comparing Output: 'onnx::Sigmoid_638' (dtype=float32, shape=(1, 3, 80, 80, 7)) with 'onnx::Sigmoid_638' (dtype=float32, shape=(1, 3, 80, 80, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_638 | Stats: mean=-1.5821, std-dev=4.023, var=16.185, median=-0.035941, min=-13.831 at (0, 0, 69, 1, 4), max=2.5998 at (0, 0, 65, 56, 5), avg-magnitude=1.9761
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.8 , -12.2 ) |       3189 | #
                (-12.2 , -10.5 ) |      12571 | #####
                (-10.5 , -8.9  ) |       3174 | #
                (-8.9  , -7.26 ) |        265 | 
                (-7.26 , -5.62 ) |          1 | 
                (-5.62 , -3.97 ) |          0 | 
                (-3.97 , -2.33 ) |         14 | 
                (-2.33 , -0.686) |       9883 | ###
                (-0.686, 0.957 ) |      99604 | ########################################
                (0.957 , 2.6   ) |       5699 | ##
[I]         onnxrt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_638 | Stats: mean=-1.5821, std-dev=4.023, var=16.185, median=-0.035944, min=-13.831 at (0, 0, 69, 1, 4), max=2.5998 at (0, 0, 65, 56, 5), avg-magnitude=1.9761
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.8 , -12.2 ) |       3189 | #
                (-12.2 , -10.5 ) |      12571 | #####
                (-10.5 , -8.9  ) |       3174 | #
                (-8.9  , -7.26 ) |        265 | 
                (-7.26 , -5.62 ) |          1 | 
                (-5.62 , -3.97 ) |          0 | 
                (-3.97 , -2.33 ) |         14 | 
                (-2.33 , -0.686) |       9883 | ###
                (-0.686, 0.957 ) |      99604 | ########################################
                (0.957 , 2.6   ) |       5699 | ##
[I]         Error Metrics: onnx::Sigmoid_638
[I]             Minimum Required Tolerance: elemwise error | [abs=6.9618e-05] OR [rel=1.4264] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=5.3915e-06, std-dev=6.0069e-06, var=3.6082e-11, median=3.5465e-06, min=0 at (0, 0, 0, 4, 5), max=6.9618e-05 at (0, 1, 44, 42, 4), avg-magnitude=5.3915e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 6.96e-06) |     101071 | ########################################
                    (6.96e-06, 1.39e-05) |      22680 | ########
                    (1.39e-05, 2.09e-05) |       6195 | ##
                    (2.09e-05, 2.78e-05) |       2783 | #
                    (2.78e-05, 3.48e-05) |       1055 | 
                    (3.48e-05, 4.18e-05) |        382 | 
                    (4.18e-05, 4.87e-05) |        160 | 
                    (4.87e-05, 5.57e-05) |         42 | 
                    (5.57e-05, 6.27e-05) |         21 | 
                    (6.27e-05, 6.96e-05) |         11 | 
[I]             Relative Difference | Stats: mean=7.0141e-05, std-dev=0.0041236, var=1.7004e-05, median=7.4882e-06, min=0 at (0, 0, 0, 4, 5), max=1.4264 at (0, 1, 12, 7, 0), avg-magnitude=7.0141e-05
[I]                 ---- Histogram ----
                    Bin Range      |  Num Elems | Visualization
                    (0    , 0.143) |     134397 | ########################################
                    (0.143, 0.285) |          2 | 
                    (0.285, 0.428) |          0 | 
                    (0.428, 0.571) |          0 | 
                    (0.571, 0.713) |          0 | 
                    (0.713, 0.856) |          0 | 
                    (0.856, 0.998) |          0 | 
                    (0.998, 1.14 ) |          0 | 
                    (1.14 , 1.28 ) |          0 | 
                    (1.28 , 1.43 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'onnx::Sigmoid_977' (dtype=float32, shape=(1, 3, 40, 40, 7)) with 'onnx::Sigmoid_977' (dtype=float32, shape=(1, 3, 40, 40, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_977 | Stats: mean=-1.3664, std-dev=3.6837, var=13.57, median=-0.054345, min=-13.749 at (0, 2, 39, 12, 4), max=2.6531 at (0, 2, 26, 5, 5), avg-magnitude=1.9664
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.7 , -12.1 ) |         45 | 
                (-12.1 , -10.5 ) |       1769 | ###
                (-10.5 , -8.83 ) |       2720 | #####
                (-8.83 , -7.19 ) |        266 | 
                (-7.19 , -5.55 ) |          0 | 
                (-5.55 , -3.91 ) |          0 | 
                (-3.91 , -2.27 ) |        103 | 
                (-2.27 , -0.627) |       4179 | ########
                (-0.627, 1.01  ) |      20353 | ########################################
                (1.01  , 2.65  ) |       4165 | ########
[I]         onnxrt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_977 | Stats: mean=-1.3664, std-dev=3.6837, var=13.57, median=-0.054349, min=-13.749 at (0, 2, 39, 12, 4), max=2.6531 at (0, 2, 26, 5, 5), avg-magnitude=1.9664
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-13.7 , -12.1 ) |         45 | 
                (-12.1 , -10.5 ) |       1769 | ###
                (-10.5 , -8.83 ) |       2720 | #####
                (-8.83 , -7.19 ) |        266 | 
                (-7.19 , -5.55 ) |          0 | 
                (-5.55 , -3.91 ) |          0 | 
                (-3.91 , -2.27 ) |        103 | 
                (-2.27 , -0.627) |       4179 | ########
                (-0.627, 1.01  ) |      20353 | ########################################
                (1.01  , 2.65  ) |       4165 | ########
[I]         Error Metrics: onnx::Sigmoid_977
[I]             Minimum Required Tolerance: elemwise error | [abs=4.2915e-05] OR [rel=0.47712] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.7101e-06, std-dev=4.0824e-06, var=1.6666e-11, median=2.414e-06, min=0 at (0, 0, 0, 30, 3), max=4.2915e-05 at (0, 0, 21, 11, 4), avg-magnitude=3.7101e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 4.29e-06) |      23717 | ########################################
                    (4.29e-06, 8.58e-06) |       6467 | ##########
                    (8.58e-06, 1.29e-05) |       2033 | ###
                    (1.29e-05, 1.72e-05) |        723 | #
                    (1.72e-05, 2.15e-05) |        412 | 
                    (2.15e-05, 2.57e-05) |        140 | 
                    (2.57e-05, 3e-05   ) |         72 | 
                    (3e-05   , 3.43e-05) |         26 | 
                    (3.43e-05, 3.86e-05) |          6 | 
                    (3.86e-05, 4.29e-05) |          4 | 
[I]             Relative Difference | Stats: mean=5.341e-05, std-dev=0.0026877, var=7.2235e-06, median=3.9517e-06, min=0 at (0, 0, 0, 30, 3), max=0.47712 at (0, 2, 29, 18, 0), avg-magnitude=5.341e-05
[I]                 ---- Histogram ----
                    Bin Range        |  Num Elems | Visualization
                    (0     , 0.0477) |      33597 | ########################################
                    (0.0477, 0.0954) |          2 | 
                    (0.0954, 0.143 ) |          0 | 
                    (0.143 , 0.191 ) |          0 | 
                    (0.191 , 0.239 ) |          0 | 
                    (0.239 , 0.286 ) |          0 | 
                    (0.286 , 0.334 ) |          0 | 
                    (0.334 , 0.382 ) |          0 | 
                    (0.382 , 0.429 ) |          0 | 
                    (0.429 , 0.477 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'onnx::Sigmoid_1316' (dtype=float32, shape=(1, 3, 20, 20, 7)) with 'onnx::Sigmoid_1316' (dtype=float32, shape=(1, 3, 20, 20, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_1316 | Stats: mean=-1.261, std-dev=3.3875, var=11.475, median=-0.12893, min=-14.461 at (0, 2, 19, 0, 4), max=3.371 at (0, 2, 12, 3, 5), avg-magnitude=1.9811
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-14.5 , -12.7 ) |         32 | 
                (-12.7 , -10.9 ) |         72 | 
                (-10.9 , -9.11 ) |        480 | #####
                (-9.11 , -7.33 ) |        496 | #####
                (-7.33 , -5.55 ) |        120 | #
                (-5.55 , -3.76 ) |          0 | 
                (-3.76 , -1.98 ) |        253 | ##
                (-1.98 , -0.195) |       2468 | ##########################
                (-0.195, 1.59  ) |       3738 | ########################################
                (1.59  , 3.37  ) |        741 | #######
[I]         onnxrt-runner-N0-09/14/22-13:32:43: onnx::Sigmoid_1316 | Stats: mean=-1.261, std-dev=3.3875, var=11.475, median=-0.12893, min=-14.461 at (0, 2, 19, 0, 4), max=3.371 at (0, 2, 12, 3, 5), avg-magnitude=1.9811
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (-14.5 , -12.7 ) |         32 | 
                (-12.7 , -10.9 ) |         72 | 
                (-10.9 , -9.11 ) |        480 | #####
                (-9.11 , -7.33 ) |        496 | #####
                (-7.33 , -5.55 ) |        120 | #
                (-5.55 , -3.76 ) |          0 | 
                (-3.76 , -1.98 ) |        253 | ##
                (-1.98 , -0.195) |       2468 | ##########################
                (-0.195, 1.59  ) |       3738 | ########################################
                (1.59  , 3.37  ) |        741 | #######
[I]         Error Metrics: onnx::Sigmoid_1316
[I]             Minimum Required Tolerance: elemwise error | [abs=3.767e-05] OR [rel=0.11714] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.1082e-06, std-dev=3.6633e-06, var=1.342e-11, median=1.8477e-06, min=0 at (0, 0, 0, 7, 6), max=3.767e-05 at (0, 1, 9, 2, 4), avg-magnitude=3.1082e-06
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 3.77e-06) |       6017 | ########################################
                    (3.77e-06, 7.53e-06) |       1492 | #########
                    (7.53e-06, 1.13e-05) |        531 | ###
                    (1.13e-05, 1.51e-05) |        208 | #
                    (1.51e-05, 1.88e-05) |         85 | 
                    (1.88e-05, 2.26e-05) |         47 | 
                    (2.26e-05, 2.64e-05) |         14 | 
                    (2.64e-05, 3.01e-05) |          1 | 
                    (3.01e-05, 3.39e-05) |          2 | 
                    (3.39e-05, 3.77e-05) |          3 | 
[I]             Relative Difference | Stats: mean=4.685e-05, std-dev=0.0015497, var=2.4015e-06, median=2.2267e-06, min=0 at (0, 0, 0, 7, 6), max=0.11714 at (0, 0, 7, 16, 1), avg-magnitude=4.685e-05
[I]                 ---- Histogram ----
                    Bin Range        |  Num Elems | Visualization
                    (0     , 0.0117) |       8397 | ########################################
                    (0.0117, 0.0234) |          1 | 
                    (0.0234, 0.0351) |          0 | 
                    (0.0351, 0.0469) |          0 | 
                    (0.0469, 0.0586) |          0 | 
                    (0.0586, 0.0703) |          0 | 
                    (0.0703, 0.082 ) |          1 | 
                    (0.082 , 0.0937) |          0 | 
                    (0.0937, 0.105 ) |          0 | 
                    (0.105 , 0.117 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[I]     Comparing Output: 'output' (dtype=float32, shape=(1, 25200, 7)) with 'output' (dtype=float32, shape=(1, 25200, 7))
[I]     Tolerance: [abs=1e-05, rel=1e-05] | Checking elemwise error
[I]         trt-runner-N0-09/14/22-13:32:43: output | Stats: mean=106.78, std-dev=173.18, var=29992, median=18.356, min=5.2426e-07 at (0, 25180, 4), max=637.66 at (0, 2639, 0), avg-magnitude=106.78
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (5.24e-07, 63.8) |     122026 | ########################################
                (63.8    , 128 ) |       9459 | ###
                (128     , 191 ) |       7071 | ##
                (191     , 255 ) |       5742 | #
                (255     , 319 ) |       5580 | #
                (319     , 383 ) |       5685 | #
                (383     , 446 ) |       5435 | #
                (446     , 510 ) |       5163 | #
                (510     , 574 ) |       5120 | #
                (574     , 638 ) |       5119 | #
[I]         onnxrt-runner-N0-09/14/22-13:32:43: output | Stats: mean=106.78, std-dev=173.18, var=29992, median=18.356, min=5.3644e-07 at (0, 25180, 4), max=637.66 at (0, 2639, 0), avg-magnitude=106.78
[I]             ---- Histogram ----
                Bin Range        |  Num Elems | Visualization
                (5.24e-07, 63.8) |     122026 | ########################################
                (63.8    , 128 ) |       9459 | ###
                (128     , 191 ) |       7071 | ##
                (191     , 255 ) |       5742 | #
                (255     , 319 ) |       5580 | #
                (319     , 383 ) |       5685 | #
                (383     , 446 ) |       5435 | #
                (446     , 510 ) |       5163 | #
                (510     , 574 ) |       5120 | #
                (574     , 638 ) |       5119 | #
[I]         Error Metrics: output
[I]             Minimum Required Tolerance: elemwise error | [abs=0.002594] OR [rel=0.077809] (requirements may be lower if both abs/rel tolerances are set)
[I]             Absolute Difference | Stats: mean=3.569e-05, std-dev=9.3724e-05, var=8.7842e-09, median=1.5497e-06, min=0 at (0, 4, 0), max=0.002594 at (0, 23419, 3), avg-magnitude=3.569e-05
[I]                 ---- Histogram ----
                    Bin Range            |  Num Elems | Visualization
                    (0       , 0.000259) |     171144 | ########################################
                    (0.000259, 0.000519) |       3850 | 
                    (0.000519, 0.000778) |        991 | 
                    (0.000778, 0.00104 ) |        251 | 
                    (0.00104 , 0.0013  ) |         93 | 
                    (0.0013  , 0.00156 ) |         40 | 
                    (0.00156 , 0.00182 ) |         19 | 
                    (0.00182 , 0.00208 ) |          8 | 
                    (0.00208 , 0.00233 ) |          1 | 
                    (0.00233 , 0.00259 ) |          3 | 
[I]             Relative Difference | Stats: mean=0.00037051, std-dev=0.0016869, var=2.8458e-06, median=1.2479e-06, min=0 at (0, 4, 0), max=0.077809 at (0, 24800, 4), avg-magnitude=0.00037051
[I]                 ---- Histogram ----
                    Bin Range          |  Num Elems | Visualization
                    (0      , 0.00778) |     174425 | ########################################
                    (0.00778, 0.0156 ) |       1618 | 
                    (0.0156 , 0.0233 ) |        267 | 
                    (0.0233 , 0.0311 ) |         61 | 
                    (0.0311 , 0.0389 ) |         16 | 
                    (0.0389 , 0.0467 ) |          6 | 
                    (0.0467 , 0.0545 ) |          2 | 
                    (0.0545 , 0.0622 ) |          3 | 
                    (0.0622 , 0.07   ) |          1 | 
                    (0.07   , 0.0778 ) |          1 | 
[E]         FAILED | Difference exceeds tolerance (rel=1e-05, abs=1e-05)
[E]     FAILED | Mismatched outputs: ['onnx::Sigmoid_638', 'onnx::Sigmoid_977', 'onnx::Sigmoid_1316', 'output']
[!] FAILED | Command: /usr/local/bin/polygraphy run folded.onnx --trt --onnxrt

Am I doing something wrong or are some of the layers not accurately converted because of some incorrect casting which causes different outputs?

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!