in installed the solution exactly according
1. Text Generation - Jetson Generative AI Playground.
I can call the website but no model can be selected. where do i get the model from, hugging face ? or any error in the installation?
Image generation runs on Jetson Orin Nano 8GB nearly not, because of limited
RAM, any solution?
Do you use jetson-containers and open the http://<IP_ADDRESS>:7860?
If yes, could you share a screenshot to show the issue?
For the memory issue, have you tried the Pre-setup steps to add some swap memory?
yes im using http://<IP_ADDRESS>:7860 and yes we did pre setup.
will provide a screenshot later today, thx for your fast answer
Hi @alexander102, yes you can download the models from HuggingFace, the text-generation-webui was compiled with GPU-accelerated loaders for many types of models (HF transformers, AutoGPTQ, exllama, llama.cpp, ect)
You can find more info about downloading models on this page (and the oobabooga github)
For Orin Nano, I would try using https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML (
llama-2-7b-chat.ggmlv3.q4_0.bin is what I’ve tried)