I cannot run LLaVa with L4T 35.5 on Jetson Orin NX 16G.
I tried referring to the following page.
https://www.jetson-ai-lab.com/tutorial_llava.html#1-chat-with-llava-using-text-generation-webui
I got the error below but I cannot figure out what means of this.
ERROR The model could not be loaded because its checkpoint file in .bin/.pt/.safetensors format could not be located.
Below is the detail of the error messages.
– Finding compatible container image for [‘text-generation-webui’]
[sudo] password for a:
dustynv/text-generation-webui:r35.4.1-cp310
-
sudo docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc
/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run
/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.so
ck --volume /home/a/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb --device /dev/i2c-0 --device /dev/i2c-1 --devi
ce /dev/i2c-2 --device /dev/i2c-3 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-6 --device /dev/i2c-7 --device /dev/i2c-8 -
-workdir=/opt/text-generation-webui dustynv/text-generation-webui:r35.4.1-cp310 python3 server.py --listen --model-dir /data/models/tex
t-generation-webui --model TheBloke_llava-v1.5-13B-GPTQ --multimodal-pipeline llava-v1.5-13b --loader autogptq --disable_exllama --verb
ose
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:124: FutureWarning: UsingTRANSFORMERS_CACHEis deprecated and will
be removed in v5 of Transformers. UseHF_HOMEinstead.
warnings.warn(
10:00:36-198813 INFO Starting Text generation web UI
10:00:36-204703 WARNING
You are potentially exposing the web UI to the entire internet without any access password.
You can create one with the “–gradio-auth” flag like this:--gradio-auth username:password Make sure to replace username:password with your own.
10:00:36-211187 INFO Loading settings from “settings.yaml”
10:00:36-216890 INFO Loading “TheBloke_llava-v1.5-13B-GPTQ”
10:00:36-268205 ERROR The model could not be loaded because its checkpoint file in .bin/.pt/.safetensors format could not be located.
10:00:36-271061 INFO Loading the extension “multimodal”
10:00:38-239843 INFO LLaVA - Loading CLIP from openai/clip-vit-large-patch14-336 as torch.float16 on cuda:0…
preprocessor_config.json: 100%|████████████████████████████████████████████████████████████████████████| 316/316 [00:00<00:00, 699kB/s]
config.json: 100%|████████████████████████████████████████████████████████████████████████████████| 4.76k/4.76k [00:00<00:00, 7.46MB/s]
pytorch_model.bin: 11%|████████▎
File “/usr/local/lib/python3.10/dist-packages/gradio/route_utils.py”, line 261, in call_process_api
output = await app.get_blocks().process_api(
File “/usr/local/lib/python3.10/dist-packages/gradio/blocks.py”, line 1786, in process_api
result = await self.call_function(
File “/usr/local/lib/python3.10/dist-packages/gradio/blocks.py”, line 1350, in call_function
prediction = await utils.async_iteration(iterator)
File “/usr/local/lib/python3.10/dist-packages/gradio/utils.py”, line 583, in async_iteration
return await iterator.anext()
File “/usr/local/lib/python3.10/dist-packages/gradio/utils.py”, line 576, in anext
return await anyio.to_thread.run_sync(
File “/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py”, line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File “/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py”, line 2144, in run_sync_in_worker_thread
return await future
File “/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py”, line 851, in run
result = context.run(func, *args)
File “/usr/local/lib/python3.10/dist-packages/gradio/utils.py”, line 559, in run_sync_iterator_async
return next(iterator)
File “/usr/local/lib/python3.10/dist-packages/gradio/utils.py”, line 742, in gen_wrapper
response = next(iterator)
File “/opt/text-generation-webui/modules/chat.py”, line 414, in generate_chat_reply_wrapper
for i, history in enumerate(generate_chat_reply(text, state, regenerate, _continue, loading_message=True, for_ui=True)):
File “/opt/text-generation-webui/modules/chat.py”, line 382, in generate_chat_reply
for history in chatbot_wrapper(text, state, regenerate=regenerate, _continue=_continue, loading_message=loading_message, for_ui=for
_ui):
File “/opt/text-generation-webui/modules/chat.py”, line 312, in chatbot_wrapper
raise ValueError(“No model is loaded! Select one in the Model tab.”)
ValueError: No model is loaded! Select one in the Model tab.