Error when running Docker VIA Microservices

Accelerated Computing Intelligent Video Analytics Visual AI Agent
1

Hello
Iam testing VIA microservice, but when I pull the Docker and run it according to the user guide (using GPT-4o for VLM and Nvidia NIM API), I encounter the following error. Can you help me?
Hardware Platform:

  • Ubuntu 20.04.6 LTS
  • GPU: A100 40GB
  • Driver Version: 535.161.07 ,
  • CUDA Version: 12.2

Fatal Python error: take_gil: PyMUTEX_LOCK(gil->switch_mutex) failed
Python runtime state: initialized

Thread 0x00007f00a1bb5480 (most recent call first):
File “”, line 241 in _call_with_frames_removed
File “”, line 1176 in create_module
File “”, line 571 in module_from_spec
File “”, line 674 in _load_unlocked
File “”, line 1006 in _find_and_load_unlocked
File “”, line 1027 in _find_and_load
File “”, line 241 in _call_with_frames_removed
File “”, line 1078 in _handle_fromlist
File “/usr/local/lib/python3.10/dist-packages/scipy/fft/_pocketfft/basic.py”, line 6 in
File “”, line 241 in _call_with_frames_removed
File “”, line 883 in exec_module
File “”, line 688 in _load_unlocked
File “”, line 1006 in _find_and_load_unlocked
File “”, line 1027 in _find_and_load
File "/usr/local/lib/python3.10/dist-packages/scipy/fft/_pocketfft/init
…

2024-08-10 16:26:18,863 PERF Summarization/BatchSummarization time = 3381.10 ms
special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 695/695 [00:00<00:00, 4.75MB/s]
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 650/650 [00:00<00:00, 5.81MB/s]
tokenizer_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.43k/1.43k [00:00<00:00, 9.84MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 712k/712k [00:00<00:00, 740kB/s]
model.onnx: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 90.4M/90.4M [00:02<00:00, 43.7MB/s]
Fetching 5 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:04<00:00, 1.19it/s]
2024-08-10 16:26:27,024 INFO Loaded Guardrails████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 90.4M/90.4M [00:02<00:00, 55.2MB/s]
2024-08-10 16:26:28,025 INFO Stopping VIA pipeline
2024-08-10 16:26:28,055 INFO Stopped VIA pipeline
2024-08-10 16:26:28,055 ERROR Failed to load VIA pipeline - Failed to load Decoder on GPU 0

3

Could you attach the guide link you referred to and your detailed operation procedures?

4

@yuweiw Thanks for your reply,
I am following the guidelines in this document: https://docs.nvidia.com/via/via_2.0_dp_user_guide.pdf#page=15.00
Docker has been pulled, and all API keys work. I tried changing the port for serving, but there is still an error.

export BACKEND_PORT=8000
export FRONTEND_PORT=9000
export NVIDIA_API_KEY=
export OPENAI_API_KEY=
docker run --rm -it --ipc=host --ulimit memlock=-1
–ulimit stack=67108864 --tmpfs /tmp:exec --name via-server
–gpus ‘“device=all”’
-p $FRONTEND_PORT:$FRONTEND_PORT
-p $BACKEND_PORT:$BACKEND_PORT
-e BACKEND_PORT=$BACKEND_PORT
-e FRONTEND_PORT=$FRONTEND_PORT
-e NVIDIA_API_KEY=$NVIDIA_API_KEY
-e OPENAI_API_KEY=$OPENAI_API_KEY
-v via-hf-cache:/tmp/huggingface
nvcr.io/metropolis/via-dp/via-engine:2.0-dp

5
  1. We recommend that you follow the Prerequisites in the Guide to setup your system.
    Ubuntu 22.04
    NVIDIA driver 535.161.08
  2. Have you installed the NVIDIA Container Toolkit according to our Guide?
6

Hi, I’m also facing the exact same error when using Vita command provided in the same document. I have Ubuntu 22.04 (Ubuntu 20.04.6 LTS) with NVIDIA driver 535.161.08 and NVIDIA Container Toolkit on A100 40GB.

7

OK. Could you try to add --privileged=true to the docker run ... command?

1 Like
8

That worked for me. Thanks.

9

That worked for me, thanks for your help.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.