Jetson ORIN NX 16 GB does not use CUDA Core

[aytekinyildizhan]

Hi,

We have Jetson Orin NX 16GB and we install pytorch and torchvision in this link

Our jetpack version is 5.1.2 so we install pytroch this version = [torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl ] because jetpack 5.1.2 is compatible with pytorch 2.1.0.

However pytorch 2.1.0 are not compatible with any torchvision.

Therefore we use torchvision v0.16.1 and it is installed successfully. Also we implemented torch.cuda.is_avaliable() then it retured TRUE.

Afterwards, we implemented the prediction in yolov8 (model.to('cuda) and device = ‘0’) for detection anomalies (such as scratch on the product), it took 250ms !! It takes too long. Even if cuda is enabled, time is too long. We expected below 100 ms.

Also we implemented 5 jetsons and 5 cameras at the same time.

What is the problem? Is that version compatible issue?

[AastaLLL]

Hi,

Have you maximized the device’s performance?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

Please verify this by monitoring the system status.
We expect GPU utilization to reach 99% when full loading.

$ sudo tegrastats

Thanks.

[aytekinyildizhan]

Hi @AastaLLL ,

We maximize the device (Jetson Orin NX 16GB - MAXN) and we implement the code (yolov8 - prediction).

Prediction result took 216ms. Again time is still too long.

I think cuda core is not working. However we have not solved it yet.

Thanks

[aytekinyildizhan]

Hi @AastaLLL ,

We solved this problem by using TensorRT.

We got 9 ms for detection.

Thanks for help.

Regards,