We are currently studying what AI could bring to our device in terms of image processing. We currently have a PC into it, but without enough computational power (no GPU).
In this context, could someone explain me the difference between the Geforce RTX (say 3090) and Jetson AGX Xavier for inference?
performances are not expressed in the same units (TFLOPS vs TOPS)
power consumption seems drastically different
Jetson AGX Xavier seems cheaper
Is Jetson Xavier more suited to embedding in a device (due to lower consumption)? Is one of the two better for training AI, and the other one more suited to inference? I am a little bit lost…
The inference performance difference between a top-of-the-line desktop GPU, and a Jetson AGX Xavier, is likely to be on the order of 20x different, similar to how the inference performance difference between the AGX Xavier and the Jetson Nano is another 20x difference.
This isn’t super surprising, both because of the significant power and cost difference, and because of the technology level difference. The Jetson Nano is now a very old GPU architecture, the AGX Xavier is a somewhat old GPU architecture, and the RTX 30 series is the currently newest available GPU architecture.
In all engineering, to solve a particular problem, you figure out how important each factor is: cost, power, time-to-market, precision, performance, etc. Then, you benchmark a few different solutions that seem to fit within the envelope. Then, you pick the solution that seems to be the best trade-off, given your needs.
Also note that there are diminishing returns in performance. The difference between 60 Hz and 240 Hz inference rate is unlikely to be particularly important to many applications, but the cost to go from one to the other might be significant. Or, similarly, you might be able to either save more power, or get better precision, if you’re currently at 240 Hz but are OK with 60 Hz (or 10 Hz, for that matter.)