DeepStream SDK 8.0 with TensorRT 10.14.1

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson
• DeepStream Version: 8.0
• JetPack Version (valid for Jetson only): 7.0
• TensorRT Version: 10.14.1
• Issue Type( questions, new requirements, bugs): questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I developed an object detection programme with nvinfer on the Jetson AGX Orin with DeepStream SDK 7.1.
I want to use Jetson Thor, so I updated to DeepStream SDK 8.0 and ran the programme, but the model only output NaN.
Upon investigation, I found that this bug can be reproduced when network-mode=2(FP16) is specified.
Furthermore, I found that the convolution layer in the model was outputting NaN values.
As both of the above results relate to TensorRT features, we considered that upgrading TensorRT from version 10.3.0 to 10.13.2 was the cause of the bug.
Additionally, as the DeepStream SDK 9.0 also uses TensorRT 10.13.2, upgrading the DeepStream SDK is considered ineffective.
I tried updating TensorRT to versions 10.13.3 and 10.14.1, and found that after updating to 10.14.1, it worked correctly with network-mode=2.

I also plan to use features of the DeepStream SDK other than nvinfer.
If I update TensorRT from version 10.13.2 to 10.14.1, will I still be able to use the DeepStream SDK’s features?

4

So have you installed DS-8.0 on Orin with JP-6.1 ? I afraid it can’t work normally.

In short, if you are still using Orin, please use JP-6.1 and DS-7.1. If you are currently using Thor, DS-9.0 and JP-7.1 GA are recommended.

Upgrading only the TRT version in JP-6.1 on Orin will not work with DS-8.0, as this involves BSP, GPU driver, etc.

5

I’m sorry if I caused any misunderstanding.

So have you installed DS-8.0 on Orin with JP-6.1 ? I afraid it can’t work normally.

No, I do not use Jetson Orin.
As I need to use DS8.0 and JP7.0 on Jetson Thor,
I currently use DS8.0 and JP7.0 on Jetson Thor.

To put it briefly, my application, which runs on Jetson Thor JP7.0, needs to use TensorRT 10.14.1 for bug fixes.
In addition, I would like to use DS 8.0 on Jetson Thor JP7.0 .
However, there is a concern that upgrading TensorRT from version 10.13.2 to 10.14.1 may prevent the DS8.0 library from correctly loading TensorRT’s symbol.

Will the DS8.0 library still load TensorRT symbols correctly even if TensorRT is upgraded to version 10.14.1?

7

Generally, we don’t recommend doing this, as we’ve only done compatibility testing with TRT-10.13. However, you could consider using the DeepStream Docker image to circumvent these issues.