Expected performance of the Jetson Orin NX 16GB

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
,
1

Hello,

I have a couple of questions concerning the expected performance of this device:
I am running a neural network on the original Jetson nano which takes roughly 800ms per image.
Now, I know that the Orin NX 16GB is advertised to have 100 TOPS of processing power, several orders of magnitude the processing power of the Nano, but I also know that performance cant be calculated linearly.
My question is, what performance should I expect from the Orin Nx 16gb on the same task? If I where to run the two instances of this task simultaneously , will I be able to achieve sub 250ms speeds per image on each instance? Can this also be achieved on the Orin Nx 8Gb ?

My other question concerns training a model on the Jetson Orin Nx.
I know that these systems are optimised for inference and perform poorly on training tasks.
However, if training needs to be done occasionally, what performance should I expect? I have a network being trained on 1050ti 4GB taking about 4-6h to complete, is there a way to know how would the Nx behave in such a scenario ? how long would it take?

I know that my questions are a little oblivious, but we’re working on a project and want to make sure that the chosen hardware is enough.

Anything helps, thanks in advance.

3

Hi,

Performance usually depends on the type of task.
The behavior of IO-bound inference (ex. small kernel convolution) will be very different compared to computation-boud inference.

So it’s recommended to give it a try directly.
But we do have some benchmark results for Orin NX for your reference:

Thanks.

4

Thanks for the reply @AastaLLL and for the benchmarks.

Do you have any advice or comments on the training part of my question ?

Also I have another question:

Is there a recommended method or approach for parallel inference with python on the Orin NX ? Any best practices?

5

Hi,

Here are some data about the training time of Jetson for your reference.
Usually, transfer learning a classification takes several hours to finish.

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.