I am writing an SDK in which, on first initialization, an onnx model is consumed and used to generate a TRT engine file. The engine file is then saved to disk, such that on subsequent initializations, the TRT engine file can be read and loaded directly instead of being regenerated.
Now I know that in the TRT documentation it mentions: “The generated plan files are not portable across platforms or TensorRT versions. Plans are specific to the exact GPU model they were built on (in addition to platforms and the TensorRT version) and must be re-targeted to the specific GPU in case you want to run them on a different GPU.”
I therefore devised a system where the GPU UUID would be appended to the name of the engine file saved to disk. When searching for the engine file, it would read the GPU UUID and search for a engine file with the corresponding UUID. This essentially prevents users of the SDK from generating on one GPU, and then copying the engine file to another GPU. However, I now would like the user to be able to generate the engine file on one GPU, and then be able to copy it over to another device which has the exact same GPU type. I am therefore wondering what attribute would be best to include in the engine file name. I was looking at the
cudaDeviceProp struct, and came across
cudaDeviceProp.name, which is described as the “SCII string identifying device”. If I removed all the spaces from this string and appended to the engine file, would it allow me to port the engine file across GPUs of the exact same type?
I guess to summarize my question really is, would two GPUs of the exact same type have the same
cudaDeviceProp.name? Would there ever be a circumstance where two GPUs of different types have the same name?