Difference outputs on ORIN AGX and Xavier NX?

Accelerated Computing CUDA CUDA Programming and Performance
, ,
1

I am inference a deep model (using tensorflow 1.15) on ORIN AGX and XAVIER NX. I turned off mix precision flag of tensorflow. However, I get a difference outputs from them. I have checked same input and after inferencing it was difference (a bit). Below is a my config hardware:

  • ORIN AGX:
    Jetpack 5.0.2
    CUDA 11.4
    tensorflow version: 1.15.5+nv22.11

  • Xavier NX:
    Jetpack 4.6.1
    CUDA 10.2
    tensorflow version: 1.15.5+nv21.12

Please tell me why I got difference outputs!
Thank you

2

This is mostly floating-point computation, correct? It also appears that you are using different library versions and different toolchain versions across the two platforms.

The numerical properties of library routines can change between software versions. A solver algorithm may have changed, for example, leading to slightly different results. Different compiler versions may lead to a slight modification in the order of floating-point operations, and finite-precision floating-point arithmetic is not associative like math (i.e. (a + b) + c can generate a different result than a + (b + c)), so this could also affect your final results.

So unless the differences are fairly dramatic, this is unlikely something you need to be worried about. For your test framework, you may want to adopt a high-precision reference and then decide on a numerical tolerance to that reference. That way small changes to the functionality under test won’t raise concerns until they exceed the tolerance threshold.