Why trt model returns bunch of Nan values?

I am transferring my onnx model to trt model, after testing the outputs of the models i found that trtmodels returns plenty of NaN values.

Here are the outputs from both models. The part values are fine, but I still got plenty of NaN values.


[array([[[3.6024790e+00, 1.9787143e+00, 2.4665287e+00, 2.4537530e+00,
          6.6367686e-03, 5.0844550e-03],
         [1.2190340e+00, 1.4738089e+00, 1.2516123e+00, 1.9088440e+00,
          8.5598230e-03, 5.2941740e-03],
         [2.4905281e+00, 1.4704772e+00, 1.7078139e+00, 1.3909163e+00,
          1.7221481e-02, 8.3563328e-03],
         [5.0809355e+00, 4.6628089e+00, 5.4717007e+00, 2.9312749e+00,
          6.7818165e-03, 2.8427649e-01],
         [5.8280797e+00, 4.6378379e+00, 3.4127855e+00, 2.3493521e+00,
          2.3016751e-02, 4.6690524e-02],
         [4.9988794e+00, 4.4283361e+00, 5.8102098e+00, 2.5537195e+00,
          1.4624715e-02, 5.5993736e-02]]], dtype=float32)]


>>> trt_infer = TensorRTInfer(trtpath)
>>> trt_infer.infer(b)
array([[[3.625     , 1.9707031 , 2.4804688 , 2.5175781 , 0.00651169,
        [1.2158203 , 1.4755859 , 1.2509766 , 1.9121094 , 0.00841522,
        [2.4882812 , 1.4541016 , 1.7050781 , 1.3740234 , 0.01737976,
        [       nan,        nan,        nan,        nan,        nan,
        [       nan,        nan,        nan,        nan,        nan,
        [       nan,        nan,        nan,        nan,        nan,
                nan]]], dtype=float32)

Here is the command i used to transferring model:

./trtexec --onnx=sample_test_fixed_batch_2.onnx --saveEngine=sample_test_fixed_batch_2.trt --workspace=4096 --fp16 --verbose

I also uploaded both of the models
sample_test_fixed_batch_2.onnx (4.0 MB)
sample_test_fixed_batch_2.trt (3.1 MB)

Dear @QQQQ,
Could you share the jetpack version and used python code snippet for reproduce on our end.

HI @SivaRamaKrishnaNV
Jetpack 4.4 and attach is the code
nvidia_sample.py (4.9 KB)

Dear @QQQQ,
Is it possible to test on latest Jetpack release and confirm?

No, i haven’t tried it on jep 4.6 , since all the other service all build based on 4.4

Please try with the latest version. Thanks