# Issue Running VSS on Jetson Thor

**URL:** <https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567>\
**Category:** Visual AI Agent\
**Tags:** cosmos, nim, blueprints\
**Created:** [December 23, 2025, 3:40pm UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567 "2025-12-23T15:40:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![abdelrahman.hosny](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/abdelrahman.hosny/32/458961_2.png) [@abdelrahman.hosny](https://forums.developer.nvidia.com/u/abdelrahman.hosny)\
**Post date:** [December 23, 2025, 3:40pm UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567/1 "2025-12-23T15:40:45Z")

</div>

- Hardware Platform (GPU model and numbers): Jetson Thor dev kit

- Ubuntu Version: Ubuntu 24.04

- NVIDIA GPU Driver Version (valid for GPU only): 580.00

- Issue Type( questions, new requirements, bugs): bugs

- How to reproduce the issue:

We are running VSS demo on a Jetson Thor device using the `remote_llm_deployment`

We are following resources from, deploying using docker compose:

- [https://github.dev/NVIDIA-AI-Blueprints/video-search-and-summarization](https://github.dev/NVIDIA-AI-Blueprints/video-search-and-summarization)
- [Deploy Using Docker Compose ARM — Video Search and Summarization Agent](https://docs.nvidia.com/vss/latest/content/vss_dep_docker_compose_arm.html)

To reproduce: `docker compose up` using the below`.env` file.

The UI on port 9100 is not opening.

```auto
docker logs -f remote_llm_deployment-via-server-1
GPU has 2 decode engines
Free GPU memory is [N/A] MiB
/opt/nvidia/via/start_via.sh: line 72: [: [N/A]: integer expression expected
Total GPU memory is 125772 MiB per GPU
Auto-selecting VLM Batch Size to 128
Using cosmos-reason1
Starting VIA server in release mode
2025-12-23 15:03:44,014 INFO Initializing VIA Stream Handler
INFO: Started server process [160]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:60000 (Press CTRL+C to quit)
2025-12-23 15:14:57,173 INFO Loaded Guardrails
2025-12-23 15:14:57,177 INFO {'gdino_engine': '/root/.via/ngc_model_cache//cv_pipeline_models/swin.fp16.engine', 'tracker_config': '/tmp/via_tracker_config.yml', 'inference_interval': 1}
2025-12-23 15:14:57,178 INFO Initializing VLM pipeline
2025-12-23 15:14:57,188 INFO Have peer access: True
2025-12-23 15:14:57,189 INFO Using model cached at /root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic
2025-12-23 15:14:57,190 INFO GPUs per VLM instance: 1
2025-12-23 15:14:57,190 INFO num_vlm_procs set to 1
2025-12-23 15:15:02,983 INFO Initializing VlmProcess-0
2025-12-23 15:15:02,983 INFO Initializing DecoderProcess-0
2025-12-23 15:15:05,111 INFO Cosmos Reason1 default system prompt: Please provide captions of all the events in the video with timestamps using the following format: <start time> <end time> caption of event 1.
<start time> <end time> caption of event 2.
At each frame, the timestamp is embedded at the bottom of the video. You need to extract the timestamp and answer the user question.
2025-12-23 15:15:05,111 INFO Using VLLM model for cosmos-reason1
2025-12-23 15:15:05,178 INFO Warmup DecoderProcess-0
/bin/dash: 1: lsmod: not found
Opening in BLOCKING MODE 
2025-12-23 15:15:05,583 INFO Video stream found.
Opening in BLOCKING MODE 
2025-12-23 15:15:05,903 INFO Video stream found.
Opening in BLOCKING MODE 
2025-12-23 15:15:06,112 INFO Video stream found.
Opening in BLOCKING MODE 
2025-12-23 15:15:06,331 INFO Video stream found.
Opening in BLOCKING MODE 
2025-12-23 15:15:06,568 INFO Video stream found.
Opening in BLOCKING MODE 
2025-12-23 15:15:06,820 INFO Video stream found.
2025-12-23 15:15:06,954 INFO Warmup DecoderProcess-0 done
2025-12-23 15:15:06,955 INFO Initialized DecoderProcess-0
INFO 12-23 15:15:07 [__init__.py:244] Automatically detected platform cuda.
2025-12-23 15:15:10,539 INFO Initializing Cosmos-Reason1-7B from: /root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic
INFO 12-23 15:15:21 [config.py:841] This model supports multiple tasks: {'embed', 'classify', 'generate', 'reward'}. Defaulting to 'generate'.
INFO 12-23 15:15:21 [config.py:1472] Using max model len 20480
INFO 12-23 15:15:21 [config.py:2285] Chunked prefill is enabled with max_num_batched_tokens=5120.
INFO 12-23 15:15:28 [__init__.py:244] Automatically detected platform cuda.
INFO 12-23 15:15:30 [core.py:526] Waiting for init message from front-end.
INFO 12-23 15:15:30 [core.py:69] Initializing a V1 LLM engine (v0.9.2+03bb1cdd.nv25.08.post1-stage-alexandrem-vss) with config: model='/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic', speculative_config=None, tokenizer='/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=20480, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":null}
[W1223 15:15:31.002821246 ProcessGroupNCCL.cpp:990] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
INFO 12-23 15:15:31 [parallel_state.py:1076] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
INFO 12-23 15:15:31 [topk_topp_sampler.py:49] Using FlashInfer for top-p & top-k sampling.
INFO 12-23 15:15:31 [gpu_model_runner.py:1770] Starting to load model /root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic...
INFO 12-23 15:15:32 [gpu_model_runner.py:1775] Loading model from scratch...
INFO 12-23 15:15:32 [cuda.py:284] Using Flash Attention backend on V1 engine.
Loading safetensors checkpoint shards: 100% 3/3 [00:03<00:00, 1.22s/it]
INFO 12-23 15:15:36 [default_loader.py:272] Loading weights took 3.79 seconds
INFO 12-23 15:15:37 [gpu_model_runner.py:1801] Model loading took 9.5227 GiB and 4.526460 seconds
INFO 12-23 15:15:37 [gpu_model_runner.py:2238] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 video items of the maximum feature size.
[rank0]:W1223 15:15:44.967000 71821 torch/_inductor/utils.py:1416] [3/0] Not enough SMs to use max_autotune_gemm mode
AUTOTUNE addmm(65536x3840, 65536x1280, 1280x3840)
strides: [0, 1], [1280, 1], [1, 1280]
dtypes: torch.bfloat16, torch.bfloat16, torch.bfloat16
  bias_addmm 5.8386 ms 100.0% 
  addmm 11.7133 ms 49.8% 
SingleProcess AUTOTUNE benchmarking takes 0.3072 seconds and 0.0002 seconds precompiling for 2 choices
AUTOTUNE addmm(16384x3584, 16384x5120, 5120x3584)
strides: [0, 1], [5120, 1], [1, 5120]
dtypes: torch.bfloat16, torch.bfloat16, torch.bfloat16
  bias_addmm 9.4843 ms 100.0% 
  addmm 12.0674 ms 78.6% 
SingleProcess AUTOTUNE benchmarking takes 4.9629 seconds and 0.0002 seconds precompiling for 2 choices
INFO 12-23 15:16:55 [backends.py:506] vLLM's torch.compile cache is disabled.
INFO 12-23 15:16:55 [backends.py:519] Dynamo bytecode transform time: 5.59 s
INFO 12-23 15:16:57 [backends.py:181] Cache the graph of shape None for later use
INFO 12-23 15:17:21 [backends.py:193] Compiling a graph for general shape takes 25.76 s
INFO 12-23 15:17:23 [monitor.py:34] torch.compile takes 31.35 s in total
INFO 12-23 15:19:04 [gpu_worker.py:232] Available KV cache memory: 32.58 GiB
INFO 12-23 15:19:04 [kv_cache_utils.py:716] GPU KV cache size: 610,080 tokens
INFO 12-23 15:19:04 [kv_cache_utils.py:720] Maximum concurrency for 20,480 tokens per request: 29.79x
Capturing CUDA graph shapes: 100% 67/67 [00:48<00:00, 1.39it/s]
INFO 12-23 15:19:53 [gpu_model_runner.py:2326] Graph capturing finished in 48 secs, took 0.72 GiB
INFO 12-23 15:19:53 [core.py:172] init engine (profile, create kv cache, warmup model) took 256.49 seconds
INFO 12-23 15:19:54 [loggers.py:137] Engine 000: vllm cache_config_info with initialization after num_gpu_blocks is: 38130
2025-12-23 15:19:55,472 INFO Max batch size: 128
2025-12-23 15:19:55,473 INFO Cosmos Reason1 VLLM model initialized successfully
2025-12-23 15:19:55,473 INFO Warmup VlmProcess-0
2025-12-23 15:19:55,474 INFO Starting model warmup...
INFO 12-23 15:20:01 [async_llm.py:270] Added request 453fd9f1-35c1-465a-b721-446badef0216.
ERROR 12-23 15:20:15 [dump_input.py:69] Dumping input data for V1 LLM engine (v0.9.2+03bb1cdd.nv25.08.post1-stage-alexandrem-vss) with config: model='/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic', speculative_config=None, tokenizer='/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=20480, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic/.vllm/torch_compile_cache/8a280991cc","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":"/root/.via/ngc_model_cache/nim_nvidia_cosmos-reason1-7b_1.1-fp8-dynamic/.vllm/torch_compile_cache/8a280991cc/rank_0_0/backbone"}, 
ERROR 12-23 15:20:15 [dump_input.py:76] Dumping scheduler output for model execution: SchedulerOutput(scheduled_new_reqs=[NewRequestData(req_id=453fd9f1-35c1-465a-b721-446badef0216,prompt_token_ids_len=172,mm_inputs=[{'video_grid_thw': tensor([[4, 10, 8]]), 'second_per_grid_ts': tensor([1.], dtype=torch.bfloat16), 'pixel_values_videos': tensor([[-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688],
ERROR 12-23 15:20:15 [dump_input.py:76] [-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688],
ERROR 12-23 15:20:15 [dump_input.py:76] [-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688],
ERROR 12-23 15:20:15 [dump_input.py:76] ...,
ERROR 12-23 15:20:15 [dump_input.py:76] [-1.7891, -1.7891, -1.7891, ..., -1.1797, -1.4766, -1.4766],
ERROR 12-23 15:20:15 [dump_input.py:76] [-1.7891, -1.7891, -1.7891, ..., -1.4766, -1.4766, -1.4766],
ERROR 12-23 15:20:15 [dump_input.py:76] [-1.7891, -1.7891, -1.7891, ..., -1.4766, -1.4766, -1.4766]],
ERROR 12-23 15:20:15 [dump_input.py:76] dtype=torch.bfloat16)}],mm_hashes=['de68b4624477db76fab93e802e2e6a6553ebbd3a2188863730c5b95b7ef73cfc'],mm_positions=[PlaceholderRange(offset=86, length=80, is_embed=None)],sampling_params=SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.1, temperature=0.7, top_p=0.9, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[151643], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=50, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None, extra_args=None),block_ids=([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],),num_computed_tokens=0,lora_request=None)], scheduled_cached_reqs=CachedRequestData(req_ids=[], resumed_from_preemption=[], new_token_ids=[], new_block_ids=[], num_computed_tokens=[]), num_scheduled_tokens={453fd9f1-35c1-465a-b721-446badef0216: 172}, total_num_scheduled_tokens=172, scheduled_spec_decode_tokens={}, scheduled_encoder_inputs={453fd9f1-35c1-465a-b721-446badef0216: [0]}, num_common_prefix_blocks=[11], finished_req_ids=[], free_encoder_input_ids=[], structured_output_request_ids={}, grammar_bitmask=null, kv_connector_metadata=null)
ERROR 12-23 15:20:15 [dump_input.py:79] Dumping scheduler stats: SchedulerStats(num_running_reqs=1, num_waiting_reqs=0, kv_cache_usage=0.00031471282454764715, prefix_cache_stats=PrefixCacheStats(reset=False, requests=1, queries=172, hits=0), spec_decoding_stats=None, num_corrupted_reqs=0)
ERROR 12-23 15:20:15 [core.py:588] EngineCore encountered a fatal error.
ERROR 12-23 15:20:15 [core.py:588] Traceback (most recent call last):
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 579, in run_engine_core
ERROR 12-23 15:20:15 [core.py:588] engine_core.run_busy_loop()
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 606, in run_busy_loop
ERROR 12-23 15:20:15 [core.py:588] self._process_engine_step()
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 631, in _process_engine_step
ERROR 12-23 15:20:15 [core.py:588] outputs, model_executed = self.step_fn()
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 235, in step
ERROR 12-23 15:20:15 [core.py:588] model_output = self.execute_model(scheduler_output)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 221, in execute_model
ERROR 12-23 15:20:15 [core.py:588] raise err
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 212, in execute_model
ERROR 12-23 15:20:15 [core.py:588] return self.model_executor.execute_model(scheduler_output)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute_model
ERROR 12-23 15:20:15 [core.py:588] output = self.collective_rpc("execute_model",
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 12-23 15:20:15 [core.py:588] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/utils/ __init__.py", line 2736, in run_method
ERROR 12-23 15:20:15 [core.py:588] return func(*args, **kwargs)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 12-23 15:20:15 [core.py:588] return func(*args, **kwargs)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 308, in execute_model
ERROR 12-23 15:20:15 [core.py:588] output = self.model_runner.execute_model(scheduler_output,
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 12-23 15:20:15 [core.py:588] return func(*args, **kwargs)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1321, in execute_model
ERROR 12-23 15:20:15 [core.py:588] self._execute_mm_encoder(scheduler_output)
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1020, in _execute_mm_encoder
ERROR 12-23 15:20:15 [core.py:588] curr_group_outputs = self.model.get_multimodal_embeddings(
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen2_5_vl.py", line 1086, in get_multimodal_embeddings
ERROR 12-23 15:20:15 [core.py:588] video_embeddings = self._process_video_input(multimodal_input)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_video_input
ERROR 12-23 15:20:15 [core.py:588] return video_embeds.split(sizes.tolist())
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] File "/usr/local/lib/python3.12/dist-packages/torch/_tensor.py", line 1053, in split
ERROR 12-23 15:20:15 [core.py:588] return torch._VF.split_with_sizes(self, split_size, dim)
ERROR 12-23 15:20:15 [core.py:588] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [core.py:588] RuntimeError: split_with_sizes expects split_sizes have only non-negative entries, but got split_sizes=[-2020506758715646411]
Process EngineCore_0:
Traceback (most recent call last):
ERROR 12-23 15:20:15 [async_llm.py:419] AsyncLLM output_handler failed.
ERROR 12-23 15:20:15 [async_llm.py:419] Traceback (most recent call last):
ERROR 12-23 15:20:15 [async_llm.py:419] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 378, in output_handler
ERROR 12-23 15:20:15 [async_llm.py:419] outputs = await engine_core.get_output_async()
ERROR 12-23 15:20:15 [async_llm.py:419] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-23 15:20:15 [async_llm.py:419] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 740, in get_output_async
ERROR 12-23 15:20:15 [async_llm.py:419] raise self._format_exception(outputs) from None
ERROR 12-23 15:20:15 [async_llm.py:419] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
  File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 590, in run_engine_core
    raise e
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 579, in run_engine_core
    engine_core.run_busy_loop()
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 606, in run_busy_loop
    self._process_engine_step()
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 631, in _process_engine_step
    outputs, model_executed = self.step_fn()
                              ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 235, in step
    model_output = self.execute_model(scheduler_output)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 221, in execute_model
    raise err
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 212, in execute_model
    return self.model_executor.execute_model(scheduler_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute_model
    output = self.collective_rpc("execute_model",
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
    answer = run_method(self.driver_worker, method, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/utils/ __init__.py", line 2736, in run_method
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 308, in execute_model
    output = self.model_runner.execute_model(scheduler_output,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1321, in execute_model
    self._execute_mm_encoder(scheduler_output)
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1020, in _execute_mm_encoder
    curr_group_outputs = self.model.get_multimodal_embeddings(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen2_5_vl.py", line 1086, in get_multimodal_embeddings
    video_embeddings = self._process_video_input(multimodal_input)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_video_input
    return video_embeds.split(sizes.tolist())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/_tensor.py", line 1053, in split
    return torch._VF.split_with_sizes(self, split_size, dim)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: split_with_sizes expects split_sizes have only non-negative entries, but got split_sizes=[-2020506758715646411]
INFO 12-23 15:20:15 [async_llm.py:345] Request 453fd9f1-35c1-465a-b721-446badef0216 failed (engine dead).
[rank0]:[W1223 15:20:16.450285082 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Process VlmProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/nvidia/via/via-engine/vlm_pipeline/process_base.py", line 245, in run
    self._warmup()
  File "/opt/nvidia/via/via-engine/vlm_pipeline/vlm_pipeline.py", line 787, in _warmup
    self._model.warmup()
  File "/opt/nvidia/via/via-engine/models/cosmos_reason1/cosmos_reason1_model.py", line 350, in warmup
    ret = ret.result()
          ^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nvidia/via/via-engine/models/cosmos_reason1/cosmos_reason1_model.py", line 310, in process_async_vllm
    output = asyncio.run_coroutine_threadsafe(generate_async(), self._event_loop).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/nvidia/via/via-engine/models/cosmos_reason1/cosmos_reason1_model.py", line 300, in generate_async
    async for output_item in self._llm.generate(
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 326, in generate
    out = q.get_nowait() or await q.get()
                            ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/output_processor.py", line 57, in get
    raise output
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 378, in output_handler
    outputs = await engine_core.get_output_async()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 740, in get_output_async
    raise self._format_exception(outputs) from None
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.

```

The `.env` file is:

```auto
export NVIDIA_API_KEY=<redacted>
export NGC_API_KEY=<redacted>

#Adjust ports if needed
export FRONTEND_PORT=9100
export BACKEND_PORT=8100

#Change default user and pass if needed 
export GRAPH_DB_USERNAME=neo4j
export GRAPH_DB_PASSWORD=password

#Update paths local paths to config files if needed. If it appears VSS is not using these configurations, then change the relative paths to absolute paths. 
export CA_RAG_CONFIG=./config.yaml
export GUARDRAILS_CONFIG=./guardrails

export ENABLE_AUDIO=false

export RIVA_ASR_SERVER_URI="grpc.nvcf.nvidia.com"
export RIVA_ASR_GRPC_PORT=443
export RIVA_ASR_SERVER_IS_NIM=true
export RIVA_ASR_SERVER_USE_SSL=true
export RIVA_ASR_SERVER_API_KEY=nvapi-*** #FIXME - api key RIVA ASR server
export RIVA_ASR_SERVER_FUNC_ID="d8dd4e9b-fbf5-4fb0-9dba-8cf436c8d965"

export DISABLE_CV_PIPELINE=false
export INSTALL_PROPRIETARY_CODECS=true	# Set to true when enabling CV

#Set VLM to Cosmos-Reason1
export VLM_MODEL_TO_USE=cosmos-reason1
export MODEL_PATH=ngc:nim/nvidia/cosmos-reason1-7b:1.1-fp8-dynamic

#Adjust misc configs if needed
export DISABLE_GUARDRAILS=false
export NVIDIA_VISIBLE_DEVICES=0 #For H100 Deployment
#export NVIDIA_VISIBLE_DEVICES=0,1,2 #For L40S Deployment

```

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [December 24, 2025, 1:47am UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567/3 "2025-12-24T01:47:49Z")

</div>

Try this script to clean up memory. I’ve tried it on agx thor and it works.

`deploy/scripts/sys_cache_cleaner.sh`

> **[Prerequisites (NVIDIA Jetson Thor) - NVIDIA Jetson Thor — Video Search and...](https://docs.nvidia.com/vss/latest/content/prereqs_thor.html#prerequisites-nvidia-jetson-thor)**
>
> VSS can be deployed on NVIDIA Jetson Thor using Docker Compose only.

---

<div class="post-metadata">

**Author:** ![abdelrahman.hosny](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/abdelrahman.hosny/32/458961_2.png) [@abdelrahman.hosny](https://forums.developer.nvidia.com/u/abdelrahman.hosny)\
**Post date:** [December 25, 2025, 4:42am UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567/4 "2025-12-25T04:42:23Z")

</div>

I left this script already running in another terminal. I stopped it, and ran it again. Then, I ran `docker compose up` and after some logging, the same error occurs. Is this something that is expected to prevent the server from coming up on :9100 ? If so, how should I debug it? I don’t seem to see the scripts on the same repo.

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [December 27, 2025, 5:56am UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567/5 "2025-12-27T05:56:59Z")

</div>

> [@abdelrahman.hosny](#):
>
> `export NGC_API_KEY=<redacted>`

The reason port 9100 is inaccessible is that the VSS server is not running correctly. Please first correctly set the `NGC_API_KEY` in your `.env` file to properly pull the cosmos-reason1 model.

You can try mounting the VSS code directory into a container to try debugging. like this topic.

> [@Alert creation overrides user-defined summarization prompts in VSS UI](https://forums.developer.nvidia.com/t/alert-creation-overrides-user-defined-summarization-prompts-in-vss-ui/348825/7):
>
> It should be included in the next release; please wait for the official announcement for the exact date. As a workaround, you can add the following configuration to your compose.yml file to override the code in the container with your local code for debugging. volumes: - xxxx/video-search-and-summarization/src/vss-engine/src:/opt/nvidia/via/via-engine

This GitHub user tried the same `remote_llm_deployment` mode and it worked after executing `sys_cache_cleaner.sh`.

> <https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/issues/67>
>
> I am trying to run the \*\*Video Search and Summarization (VSS)\*\* blueprint on \*\*J…etson Thor\*\*. However, the deployment consistently fails at the point where via-server-1 logs “\*\*Starting model warmup…\*\*”—the engine crashes shortly after.
> 
> \### Environment / Prerequisites
> \*\*Device\*\*: Jetson Thor Developer Kit
> \*\*Jetson Linux\*\*: 38.2.1
> \*\*VSS Engine Image\*\*: nvcr.io/nvidia/blueprint/vss-engine:2.4.0
> \*\*Deployment Method\*\*: Hybrid Deployment (remote\_llm\_deployment/ → docker compose up)
> 
> \### Error Logs
> 
> \> via-server-1 | 2025-12-15 10:54:42,556 INFO Cosmos Reason1 VLLM model initialized successfully
> via-server-1 | 2025-12-15 10:54:42,556 INFO Warmup VlmProcess-0
> via-server-1 | 2025-12-15 10:54:42,556 INFO Starting model warmup...
> via-server-1 | INFO 12-15 10:54:48 \[async\_llm.py:270\] Added request 35663b2b-1acd-4bfd-b80d-54e4ff3cbee7.
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:69\] Dumping input data for V1 LLM engine (v0.9.2+03bb1cdd.nv25.08.post1-stage-alexandrem-vss) with config: model='/root/.via/ngc\_model\_cache/nim\_nvidia\_cosmos-reason1-7b\_1.1-fp8-dynamic', speculative\_config=None, tokenizer='/root/.via/ngc\_model\_cache/nim\_nvidia\_cosmos-reason1-7b\_1.1-fp8-dynamic', skip\_tokenizer\_init=False, tokenizer\_mode=auto, revision=None, override\_neuron\_config={}, tokenizer\_revision=None, trust\_remote\_code=False, dtype=torch.bfloat16, max\_seq\_len=20480, download\_dir=None, load\_format=LoadFormat.AUTO, tensor\_parallel\_size=1, pipeline\_parallel\_size=1, disable\_custom\_all\_reduce=False, quantization=compressed-tensors, enforce\_eager=False, kv\_cache\_dtype=auto, device\_config=cuda, decoding\_config=DecodingConfig(backend='auto', disable\_fallback=False, disable\_any\_whitespace=False, disable\_additional\_properties=False, reasoning\_backend=''), observability\_config=ObservabilityConfig(show\_hidden\_metrics\_for\_version=None, otlp\_traces\_endpoint=None, collect\_detailed\_traces=None), seed=0, served\_model\_name=/root/.via/ngc\_model\_cache/nim\_nvidia\_cosmos-reason1-7b\_1.1-fp8-dynamic, num\_scheduler\_steps=1, multi\_step\_stream\_outputs=True, enable\_prefix\_caching=True, chunked\_prefill\_enabled=True, use\_async\_output\_proc=True, pooler\_config=None, compilation\_config={"level":3,"debug\_dump\_path":"","cache\_dir":"/root/.via/ngc\_model\_cache/nim\_nvidia\_cosmos-reason1-7b\_1.1-fp8-dynamic/.vllm/torch\_compile\_cache/8a280991cc","backend":"","custom\_ops":\[\],"splitting\_ops":\["vllm.unified\_attention","vllm.unified\_attention\_with\_output"\],"use\_inductor":true,"compile\_sizes":\[\],"inductor\_compile\_config":{"enable\_auto\_functionalized\_v2":false},"inductor\_passes":{},"use\_cudagraph":true,"cudagraph\_num\_of\_warmups":1,"cudagraph\_capture\_sizes":\[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1\],"cudagraph\_copy\_inputs":false,"full\_cuda\_graph":false,"max\_capture\_size":512,"local\_cache\_dir":"/root/.via/ngc\_model\_cache/nim\_nvidia\_cosmos-reason1-7b\_1.1-fp8-dynamic/.vllm/torch\_compile\_cache/8a280991cc/rank\_0\_0/backbone"}, 
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] Dumping scheduler output for model execution: SchedulerOutput(scheduled\_new\_reqs=\[NewRequestData(req\_id=35663b2b-1acd-4bfd-b80d-54e4ff3cbee7,prompt\_token\_ids\_len=172,mm\_inputs=\[{'pixel\_values\_videos': tensor(\[\[-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] \[-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] \[-1.7812, -1.7812, -1.7812, ..., -1.4688, -1.4688, -1.4688\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] ...,
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] \[-1.7891, -1.7891, -1.7891, ..., -1.1797, -1.4766, -1.4766\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] \[-1.7891, -1.7891, -1.7891, ..., -1.4766, -1.4766, -1.4766\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] \[-1.7891, -1.7891, -1.7891, ..., -1.4766, -1.4766, -1.4766\]\],
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:76\] dtype=torch.bfloat16), 'video\_grid\_thw': tensor(\[\[4, 10, 8\]\]), 'second\_per\_grid\_ts': tensor(\[1.\], dtype=torch.bfloat16)}\],mm\_hashes=\['de68b4624477db76fab93e802e2e6a6553ebbd3a2188863730c5b95b7ef73cfc'\],mm\_positions=\[PlaceholderRange(offset=86, length=80, is\_embed=None)\],sampling\_params=SamplingParams(n=1, presence\_penalty=0.0, frequency\_penalty=0.0, repetition\_penalty=1.1, temperature=0.7, top\_p=0.9, top\_k=0, min\_p=0.0, seed=None, stop=\[\], stop\_token\_ids=\[151643\], bad\_words=\[\], include\_stop\_str\_in\_output=False, ignore\_eos=False, max\_tokens=50, min\_tokens=0, logprobs=None, prompt\_logprobs=None, skip\_special\_tokens=True, spaces\_between\_special\_tokens=True, truncate\_prompt\_tokens=None, guided\_decoding=None, extra\_args=None),block\_ids=(\[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11\],),num\_computed\_tokens=0,lora\_request=None)\], scheduled\_cached\_reqs=CachedRequestData(req\_ids=\[\], resumed\_from\_preemption=\[\], new\_token\_ids=\[\], new\_block\_ids=\[\], num\_computed\_tokens=\[\]), num\_scheduled\_tokens={35663b2b-1acd-4bfd-b80d-54e4ff3cbee7: 172}, total\_num\_scheduled\_tokens=172, scheduled\_spec\_decode\_tokens={}, scheduled\_encoder\_inputs={35663b2b-1acd-4bfd-b80d-54e4ff3cbee7: \[0\]}, num\_common\_prefix\_blocks=\[11\], finished\_req\_ids=\[\], free\_encoder\_input\_ids=\[\], structured\_output\_request\_ids={}, grammar\_bitmask=null, kv\_connector\_metadata=null)
> via-server-1 | ERROR 12-15 10:55:02 \[dump\_input.py:79\] Dumping scheduler stats: SchedulerStats(num\_running\_reqs=1, num\_waiting\_reqs=0, kv\_cache\_usage=0.0003105750815259878, prefix\_cache\_stats=PrefixCacheStats(reset=False, requests=1, queries=172, hits=0), spec\_decoding\_stats=None, num\_corrupted\_reqs=0)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] EngineCore encountered a fatal error.
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] Traceback (most recent call last):
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 579, in run\_engine\_core
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] engine\_core.run\_busy\_loop()
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 606, in run\_busy\_loop
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] self.\_process\_engine\_step()
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 631, in \_process\_engine\_step
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] outputs, model\_executed = self.step\_fn()
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 235, in step
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] model\_output = self.execute\_model(scheduler\_output)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 221, in execute\_model
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] raise err
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 212, in execute\_model
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return self.model\_executor.execute\_model(scheduler\_output)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute\_model
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] output = self.collective\_rpc("execute\_model",
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc\_executor.py", line 57, in collective\_rpc
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] answer = run\_method(self.driver\_worker, method, args, kwargs)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/utils/\_\_init\_\_.py", line 2736, in run\_method
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return func(\*args, \*\*kwargs)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py", line 116, in decorate\_context
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return func(\*args, \*\*kwargs)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_worker.py", line 308, in execute\_model
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] output = self.model\_runner.execute\_model(scheduler\_output,
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py", line 116, in decorate\_context
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return func(\*args, \*\*kwargs)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_model\_runner.py", line 1321, in execute\_model
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] self.\_execute\_mm\_encoder(scheduler\_output)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_model\_runner.py", line 1020, in \_execute\_mm\_encoder
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] curr\_group\_outputs = self.model.get\_multimodal\_embeddings(
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/model\_executor/models/qwen2\_5\_vl.py", line 1086, in get\_multimodal\_embeddings
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] video\_embeddings = self.\_process\_video\_input(multimodal\_input)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/vllm/model\_executor/models/qwen2\_5\_vl.py", line 1045, in \_process\_video\_input
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return video\_embeds.split(sizes.tolist())
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] File "/usr/local/lib/python3.12/dist-packages/torch/\_tensor.py", line 1053, in split
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] return torch.\_VF.split\_with\_sizes(self, split\_size, dim)
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[core.py:588\] RuntimeError: split\_with\_sizes expects split\_sizes have only non-negative entries, but got split\_sizes=\[-2020506758715646411\]
> via-server-1 | Process EngineCore\_0:
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] AsyncLLM output\_handler failed.
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] Traceback (most recent call last):
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async\_llm.py", line 378, in output\_handler
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] outputs = await engine\_core.get\_output\_async()
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core\_client.py", line 740, in get\_output\_async
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] raise self.\_format\_exception(outputs) from None
> via-server-1 | ERROR 12-15 10:55:02 \[async\_llm.py:419\] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
> via-server-1 | INFO 12-15 10:55:02 \[async\_llm.py:345\] Request 35663b2b-1acd-4bfd-b80d-54e4ff3cbee7 failed (engine dead).
> via-server-1 | Traceback (most recent call last):
> via-server-1 | File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in \_bootstrap
> via-server-1 | self.run()
> via-server-1 | File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
> via-server-1 | self.\_target(\*self.\_args, \*\*self.\_kwargs)
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 590, in run\_engine\_core
> via-server-1 | raise e
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 579, in run\_engine\_core
> via-server-1 | engine\_core.run\_busy\_loop()
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 606, in run\_busy\_loop
> via-server-1 | self.\_process\_engine\_step()
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 631, in \_process\_engine\_step
> via-server-1 | outputs, model\_executed = self.step\_fn()
> via-server-1 | ^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 235, in step
> via-server-1 | model\_output = self.execute\_model(scheduler\_output)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 221, in execute\_model
> via-server-1 | raise err
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 212, in execute\_model
> via-server-1 | return self.model\_executor.execute\_model(scheduler\_output)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute\_model
> via-server-1 | output = self.collective\_rpc("execute\_model",
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc\_executor.py", line 57, in collective\_rpc
> via-server-1 | answer = run\_method(self.driver\_worker, method, args, kwargs)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/utils/\_\_init\_\_.py", line 2736, in run\_method
> via-server-1 | return func(\*args, \*\*kwargs)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py", line 116, in decorate\_context
> via-server-1 | return func(\*args, \*\*kwargs)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_worker.py", line 308, in execute\_model
> via-server-1 | output = self.model\_runner.execute\_model(scheduler\_output,
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py", line 116, in decorate\_context
> via-server-1 | return func(\*args, \*\*kwargs)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_model\_runner.py", line 1321, in execute\_model
> via-server-1 | self.\_execute\_mm\_encoder(scheduler\_output)
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu\_model\_runner.py", line 1020, in \_execute\_mm\_encoder
> via-server-1 | curr\_group\_outputs = self.model.get\_multimodal\_embeddings(
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/model\_executor/models/qwen2\_5\_vl.py", line 1086, in get\_multimodal\_embeddings
> via-server-1 | video\_embeddings = self.\_process\_video\_input(multimodal\_input)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/model\_executor/models/qwen2\_5\_vl.py", line 1045, in \_process\_video\_input
> via-server-1 | return video\_embeds.split(sizes.tolist())
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/torch/\_tensor.py", line 1053, in split
> via-server-1 | return torch.\_VF.split\_with\_sizes(self, split\_size, dim)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | RuntimeError: split\_with\_sizes expects split\_sizes have only non-negative entries, but got split\_sizes=\[-2020506758715646411\]
> via-server-1 | \[rank0\]:\[W1215 10:55:03.942084463 ProcessGroupNCCL.cpp:1538\] Warning: WARNING: destroy\_process\_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
> via-server-1 | Process VlmProcess-1:
> via-server-1 | Traceback (most recent call last):
> via-server-1 | File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in \_bootstrap
> via-server-1 | self.run()
> via-server-1 | File "/opt/nvidia/via/via-engine/vlm\_pipeline/process\_base.py", line 245, in run
> via-server-1 | self.\_warmup()
> via-server-1 | File "/opt/nvidia/via/via-engine/vlm\_pipeline/vlm\_pipeline.py", line 787, in \_warmup
> via-server-1 | self.\_model.warmup()
> via-server-1 | File "/opt/nvidia/via/via-engine/models/cosmos\_reason1/cosmos\_reason1\_model.py", line 350, in warmup
> via-server-1 | ret = ret.result()
> via-server-1 | ^^^^^^^^^^^^
> via-server-1 | File "/usr/lib/python3.12/concurrent/futures/\_base.py", line 456, in result
> via-server-1 | return self.\_\_get\_result()
> via-server-1 | ^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/lib/python3.12/concurrent/futures/\_base.py", line 401, in \_\_get\_result
> via-server-1 | raise self.\_exception
> via-server-1 | File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
> via-server-1 | result = self.fn(\*self.args, \*\*self.kwargs)
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/opt/nvidia/via/via-engine/models/cosmos\_reason1/cosmos\_reason1\_model.py", line 310, in process\_async\_vllm
> via-server-1 | output = asyncio.run\_coroutine\_threadsafe(generate\_async(), self.\_event\_loop).result()
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/lib/python3.12/concurrent/futures/\_base.py", line 456, in result
> via-server-1 | return self.\_\_get\_result()
> via-server-1 | ^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/lib/python3.12/concurrent/futures/\_base.py", line 401, in \_\_get\_result
> via-server-1 | raise self.\_exception
> via-server-1 | File "/opt/nvidia/via/via-engine/models/cosmos\_reason1/cosmos\_reason1\_model.py", line 300, in generate\_async
> via-server-1 | async for output\_item in self.\_llm.generate(
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async\_llm.py", line 326, in generate
> via-server-1 | out = q.get\_nowait() or await q.get()
> via-server-1 | ^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/output\_processor.py", line 57, in get
> via-server-1 | raise output
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async\_llm.py", line 378, in output\_handler
> via-server-1 | outputs = await engine\_core.get\_output\_async()
> via-server-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> via-server-1 | File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core\_client.py", line 740, in get\_output\_async
> via-server-1 | raise self.\_format\_exception(outputs) from None
> via-server-1 | vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
> 
> 
> 
> \*\*After discussing with colleagues, we suspect a compatibility issue with the latest vLLM version. Any guidance or insights would be greatly appreciated!\*\*

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [February 2, 2026, 9:09am UTC](https://forums.developer.nvidia.com/t/issue-running-vss-on-jetson-thor/355567/7 "2026-02-02T09:09:45Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.