Head at commit ID- 2d6187b00eaad34a4a51bf1e088baf4a600faa09
Hey Dusty, I am f…ollowing your [tutorial](https://www.jetson-ai-lab.com/tutorial_live-llava.html) on my new AGX ORIN Dev kit but when I try to run the run.sh file, it fails at the end. I will need your help to solve this, please.
Sharing the terminal output below -
```
./run.sh -e SSL_KEY=/data/key.pem -e SSL_CERT=/data/cert.pem $(./autotag local_llm) python3 -m local_llm.agents.video_query --api=mlc --verbose --model liuhaotian/llava-v1.5-7b --max-new-tokens 32 --video-input /dev/video0 --video-output Display://0 --prompt "How many fingers am I holding up?"
Namespace(disable=[''], output='/tmp/autotag', packages=['local_llm'], prefer=['local', 'registry', 'build'], quiet=False, user='dustynv', verbose=False)
-- L4T_VERSION=35.2.1 JETPACK_VERSION=5.1 CUDA_VERSION=11.4.315
-- Finding compatible container image for ['local_llm']
dustynv/local_llm:r35.3.1
localuser:root being added to access control list
+ docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /home/rajesh/Desktop/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb -e DISPLAY=:1 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth --device /dev/video0 --device /dev/video1 -e SSL_KEY=/data/key.pem -e SSL_CERT=/data/cert.pem dustynv/local_llm:r35.3.1 python3 -m local_llm.agents.video_query --api=mlc --verbose --model liuhaotian/llava-v1.5-7b --max-new-tokens 32 --video-input /dev/video0 --video-output Display://0 --prompt 'How many fingers am I holding up?'
/usr/local/lib/python3.8/dist-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
warnings.warn(
16:32:47 | DEBUG | Namespace(api='mlc', chat_template=None, debug=True, do_sample=False, log_level='debug', max_new_tokens=32, min_new_tokens=-1, model='liuhaotian/llava-v1.5-7b', prompt=['How many fingers am I holding up?'], quant=None, repetition_penalty=1.0, save_mermaid=None, system_prompt=None, temperature=0.7, top_p=0.95, video_input='/dev/video0', video_input_codec=None, video_input_framerate=None, video_input_height=None, video_input_save=None, video_input_width=None, video_output='Display://0', video_output_bitrate=None, video_output_codec=None, video_output_save=None, vision_model=None)
16:32:47 | DEBUG | subprocess 34 started
16:32:47 | DEBUG | Starting new HTTPS connection (1): huggingface.co:443
16:32:48 | DEBUG | https://huggingface.co:443 "GET /api/models/liuhaotian/llava-v1.5-7b/revision/main HTTP/1.1" 200 2444
Fetching 11 files: 100%|█████████████████████| 11/11 [00:00<00:00, 59686.09it/s]
16:32:48 | INFO | loading /data/models/huggingface/models--liuhaotian--llava-v1.5-7b/snapshots/12e054b30e8e061f423c7264bc97d4248232e965 with MLC
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/local_llm/local_llm/agents/video_query.py", line 115, in <module>
agent = VideoQuery(**vars(args)).run()
File "/opt/local_llm/local_llm/agents/video_query.py", line 22, in __init__
self.llm = ProcessProxy((lambda **kwargs: ChatQuery(model, drop_inputs=True, **kwargs)), **kwargs)
File "/opt/local_llm/local_llm/plugins/process_proxy.py", line 31, in __init__
raise RuntimeError(f"subprocess has an invalid initialization status ({init_msg['status']})")
RuntimeError: subprocess has an invalid initialization status (<class 'AssertionError'>)
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/local_llm/local_llm/plugins/process_proxy.py", line 62, in run_process
raise error
File "/opt/local_llm/local_llm/plugins/process_proxy.py", line 59, in run_process
plugin = factory(**kwargs)
File "/opt/local_llm/local_llm/agents/video_query.py", line 22, in <lambda>
self.llm = ProcessProxy((lambda **kwargs: ChatQuery(model, drop_inputs=True, **kwargs)), **kwargs)
File "/opt/local_llm/local_llm/plugins/chat_query.py", line 63, in __init__
self.model = LocalLM.from_pretrained(model, **kwargs)
File "/opt/local_llm/local_llm/local_llm.py", line 72, in from_pretrained
model = MLCModel(model_path, **kwargs)
File "/opt/local_llm/local_llm/models/mlc.py", line 58, in __init__
assert(self.device.exist) # this is needed to initialize CUDA?
AssertionError
```