Hi @daniel.widmann,
If you are using same engine with same input, TensorRT should be deterministic.
However I don’t think engine building is supposed to be deterministic as tactics are chosen based on observed runtime. If you’re outputting your log with info level, you should be able to compare tactic selection between the two engines. Since different tactics/kernels could change order of operations, you would expect floating point differences.
You can refer to the below link.
Thank you for your fast answer. It is very good to hear that TensorRT can be deterministic!
Regarding your example: It does indeed show a way to make the builder deterministic, as well, which is very interesting. In the example, custom algorithm selectors are provided to cache chosen tactics and to read them back in the next build. So, to come back to my third question ones more: Could I use this approach to cache my chosen tactics, then build the network on a different GPU using the cached tactics, and finally get a network that behaves the same on both GPUs? Or is it unavoidable that two different types of GPUs will always have small deviations in the network output?
I believe so.
Even with same tactic, the output may still have small differences across GPUs with different architecture. There is no reason in general to believe that a set of tactics valid on one GPU is valid on a different GPU.