Hello everyone,
While running the commands to setup text generation webui from LLaVA - NVIDIA Jetson AI Lab by @dusty_nv, I am getting the following error message.
Error: Couldn’t find a compatible container for text-generation-webui
I am attaching the configurations of the Jetson AGX Orin, that I am using. Could someone please help me out with this.
Thank you very much!
Hi,
Here are some suggestions for the common issues:
Please run the below command before benchmarking deep learning use case:
$ sudo nvpmodel -m 0
$ sudo jetson_clocks
Installation guide of deep learning frameworks on Jetson:
Startup deep learning tutorial:
If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.
Thanks!
Hi,
Could you share the complete log with us?
Suppose the script will start to build a container based on your environment information.
Thanks.
Hi,
Could you help to check the CUDA version in your environment?
$ ll /usr/local/cuda*
JetPack 6.1 should include CUDA 12.6 but it’s CUDA 12.1 detected in your environment.
Thanks.
Hello,
Yes, I have manually linked the “/usr/local/cuda” to CUDA 12.1, since the model( haotian-liu/LLaVA: [NeurIPS’23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.) i am trying to run requires CUDA 12.1. I have linked it back to CUDA 12.6.
Now, while running the command:
jetson-containers run --workdir=/opt/text-generation-webui $(autotag text-generation-webui) *
** python3 download-model.py --output=/data/models/text-generation-webui *
** TheBloke/llava-v1.5-13B-GPTQ**
I am getting the following error:
Is there anything I can do?
Hi,
Do you have custom Python version?
JetPack 6.1 should by default use Python 3.10.
But the error seems related to the Python3.6?
Thanks.
No, I am using Python version 3.10.
