LLM run on Jetson Orin Nano

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

Hi,

I managed to run an LLM on the Jetson Orin Nano board. Obviously not out of the box, with minimal changes to open-source components.

The model is the phi-1_5 described in arXiv:2309.05463.

Do you have guidelines or recommendations for writing a Blog article on this? with the focus on how to avoid license/EULA and other legal pitfalls?

2

Nice! I welcome you to join us in this running thread about LLMs on Jetson that we have going over here:

You should be able to run quantized llama-2-7b on Orin Nano without much issue. Some more resources that may be of interest to you:

I’m not a lawyer and this is not legal advice, personally however I would recommend using models like Llama-2 that have a commercial/permissive license.

1 Like
4

Thank you for the warm welcome :)

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.