Cross GPU optimization

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello,

I have a dev computer with a RTX2080 on Ubuntu 18.04 with TensorRT 5.0.2.
I can optimize a caffe net and create a file containing the serialized engine.

My question is : is the tensorRT engine only compatible to my RTX 2080 or can I use it on a GTX 1080 for example ? is there any way to specify the target GPU when optimizing ?

Thanks

2

Hello,
A generated TensorRT PLAN is valid for a specific GPU — more precisely, a specific CUDA Compute Capability. For example, if you generate a PLAN for an NVIDIA P4 (compute capability 6.1) you can’t use that PLAN on an NVIDIA Tesla V100 (compute capability 7.0).

RTX 2800 is SM 7.5. GTX 1080 is SM 6.1.

PLAN generation must run on the target system in order for the TensorRT engine to be optimized correctly for that system.

3

Thanks for your help.

I would need a last precision.

When you say “PLAN generation must run on the target system”, if I run the PLAN generation on my dev computer with the same OS, the same GPU SM, the same cuda and cudnn, and the same nvidia driver version, is it ok ?

Thanks

4

This should be fine.