Jetson orin nano deploying yolo with llm

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

Im deloying yolo26s/n in jetson orin nano with SLM but im running out of cuda when im using llm. im deploying it as a website and the llm is qwen 2.5 1b, and using RAG, is there way to optimize the gpu ram . I already tried the swap file , disable gui is not possible because i need to see real time in jetson. Can i ask advices

5

Hi,

Do you mean running out of memory?
Which branch do you use? Could you help provide the information with the command below?

$ cat /etc/nv_tegra_release

Please note that there are some memory-related issues in r36.4.7.
It’s recommended to use r36.5 instead.

Thanks.

6

1920×1440 504 KB

7

Hi

Could you try to set up your environment to r36.5 (JetPack 6.2.2) and test it again?

Thanks.