Jetson Generative AI Playground - Tutorial 1 - Text Generation

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

in installed the solution exactly according

text-generation-webui - NVIDIA Jetson Generative AI Lab.

I can call the website but no model can be selected. where do i get the model from, hugging face ? or any error in the installation?

Image generation runs on Jetson Orin Nano 8GB nearly not, because of limited
RAM, any solution?

Thx

3

Hi,

Do you use jetson-containers and open the http://<IP_ADDRESS>:7860?
If yes, could you share a screenshot to show the issue?

For the memory issue, have you tried the Pre-setup steps to add some swap memory?

Thanks.

4

yes im using http://<IP_ADDRESS>:7860 and yes we did pre setup.
will provide a screenshot later today, thx for your fast answer

5

Hi @alexander102, yes you can download the models from HuggingFace, the text-generation-webui was compiled with GPU-accelerated loaders for many types of models (HF transformers, AutoGPTQ, exllama, llama.cpp, ect)

You can find more info about downloading models on this page (and the oobabooga github)

For Orin Nano, I would try using https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML (llama-2-7b-chat.ggmlv3.q4_0.bin is what I’ve tried)

1 Like
6

thx Dustin

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.