Trained model giving slightly different values when tested on P100 and V100 . is there a way to make it consistent.?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi all,
i have currently trained a model using V100 GPU and my inference device is using P100. when i test it on test dataset, both the hardwares are giving slightly different values which eventually reduces my overall score on the P100 hardware. i tried to subtract the difference between the values and use the mean of it to scale my P100 prediction values . this helped me a little bit but, this is not solving my problem.
i am using Pytorch 1.8.0 for this work . is there a better way to address this problem of difference in performance due to hardware difference between Training and Inferencing environment.

thanks
yuvaram

5

Hi @yuvaramsingh94,

The generated plan files are not portable across platforms or TensorRT versions. Plans are specific to the exact GPU model they were built on (in addition to the platforms and the TensorRT version) and must be re-targeted to the specific GPU in case you want to run them on a different GPU.

Thanks

6

thanks for reply . i am currently not using TensorRT. i am just using the native pytorch for inferencing. my hardware is the main difference between training and inferencing

7

Hi @yuvaramsingh94,

In that case I will request you to raise query in below forum.

Thanks

8

thanks . will do this