tensorRT engine file can be used in different TX2 devices?

[harry_xiaye]

I have some TX2 boards, when I created one engine file on one TX2 board, I can use it in other TX2 boards, right?

I know the engine file depends on the board, but if I use the same chip(TX2), the engine file should be able to be used for all TX2 boards, right?

Thanks
Harry

[AastaLLL]

Hi,

YES. But please make sure you have the same TensorRT version for all the devices.
You can find more information in our document:
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#faq

Q: If I build the engine on one GPU and run the engine on another GPU, does this work?

A: We recommend that you don’t; however if you do, you’ll need to follow these guidelines:

1. The major, minor, and patch versions of TensorRT must match between systems. This ensures you are picking kernels that are still present and have not undergone certain optimizations or bug fixes that would change their behavior.
2. The CUDA compute capability major and minor versions must match between systems. This ensures that the same hardware features are present so the kernel does not fail to execute. An example would be mixing cards with different precision capabilities.
3. The following properties should match between systems:
   – Maximum GPU graphics clock speed
   – Maximum GPU memory clock speed
   – GPU memory bus width
   – Total GPU memory
   – GPU L2 cache size
   – SM processor count
   – Asynchronous engine count

If any of the previous properties do not match, you receive the following warning: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.

If you still want to proceed, then you should build the engine on the smallest SKU in the family because autotuner choices made on smaller GPUs generalize better.

Thanks.

[harry_xiaye]

@AastaLLL thanks your information.