How to ensure/get portable across GPU platforms?

zhaibo99 · August 31, 2020, 7:31am

When using TensorRT to build engine, can we choose not perform GPU platform related optimization, so as to ensure that the serialized plan file can be used on different GPU platforms? In short, how to ensure/get portable across GPU platforms?
Is there any way to get portable across GPU platforms when optimizing model for inference? Looking forward to the answer…

AakankshaS · August 31, 2020, 3:43pm

Hi @zhaibo99,
The generated plan file is not portable across platforms, TRT versions and GPU model.
Please refer to the below link for the same.

Thanks!

zhaibo99 · September 1, 2020, 3:37am

Thanks, hope to support cross GPU platform porting in the future.

Topic		Replies	Views
Is UFF file portable across different GPU? TensorRT	1	293	May 12, 2020
Do tensorRT plan files are portable across different GPUs which have the same type TensorRT	2	6145	August 13, 2020
Tensor-rt plan file for specific target TensorRT	3	1140	June 2, 2022
Doubts on reusing serialized engines on different platforms or TensorRT versions TensorRT	3	617	August 22, 2019
TensorRT INT8 Cache Portability TensorRT	3	851	February 20, 2020
Cross Operating System platform compatibility with the same GPU platform TensorRT tensorrt	1	433	April 11, 2022
How to get the info from the plan file? TensorRT	1	1046	July 3, 2019
Tensorrt conversion of a model for jetson nano on host device TensorRT tensorrt , jetson-inference	4	754	April 27, 2023
TRT engine across different driver version TensorRT	3	1381	December 1, 2020
TensorRT model hardware software binding problem TensorRT tensorrt , cuda	1	347	November 11, 2022

How to ensure/get portable across GPU platforms?

Related topics