Via on r36.2 dp

Accelerated Computing Intelligent Video Analytics Metropolis Microservices for Jetson
1

Hi:
I’ve installed the VIA on an AGX Orin 32G with r36.2 DP and it works pretty well. (mid of July)
And now I follow the same steps to make another copy.
It doesn’t woks at all.
There are some kinds failures as the following

  1. The models can’t be completely downloaded.
    => I did
    cd ~/vlm_conference_demo/vlm
    sudo cp ~/vlm_conference_demo/vlm/config/vlm-nginx.conf
    /opt/nvidia/jetson/services/ingress/config/

       sudo systemctl restart jetson-ingress
   sudo docker compose up -d

   Waiting for more than 1 hour, the 'vlm' folder is not be generated, I have to restart the docker
   and it starts the download of model.

  2. Even the download procedure starts, after a 8~9 hours waiting, observe the mmj_vlm log, there are some failures. it stops and shows
    InternalError: Check failed: (lib_handle_ != nullptr) is false: Failed to load dynamic shared library /data/models/mlc/dist/VILA1.5-3b-ctx4096/VILA1.5-3b-q4f1o /data/models/mlc/dist/VILA1.5-3b-ctx4096/VILA1.5-3b-q4f16_ft/VILA1.5-3b-q4f16_ft-cuda.so: file too short
    (see the attachment)

  3. Due to the above error, I replace the so file with the one that existing in the working VIA device, the system crashes all the time.

We plan to demo this in an overseas exhibition,
Could somebody help?
log.txt (52.8 KB)

3

I will check and feedback later. Seem you want to deploy VILA1.5-3b on AGX Orin 32G. right?

4

Hi Thanks for your reply:
Actually we’d like to demo [VILA1.5-13b]on AGX Orin 64G.
Due to the time consideration,
[VILA1.5-3b] on AGX Orin 32G can be one of the alternatives.

5

Please refer here for VLM: VLM Workflow — Metropolis on Jetson documentation 0.1.0 documentation
Need install JP6.0 GA based on quick start guide: Quick Start Guide — Metropolis on Jetson documentation 0.1.0 documentation

6

Hi:
Follow the r36.3 GA steps,
after deploying VLM service, Jetson shows “low disc space”.
And actually I attach a 256GB HDD to the Jetson over USB.

  1. Observe the “file browser”, the attached HDD is shown in the list.
    (Comparing to r36.2 DP, once the “jetson-storage” service starts, the disc should not show in the file browser )
  2. I tried to start up the “jetson-storage” service manually, it doesn’t work.

And finally the the size of models downloaded from huggingface is only less than 10GB.
Can you help?

7

Can you show the log of:
$ sudo systemctl restart jetson-storage
$ systemctl status jetson-storage

8

Hi:
Finally I can install my Jetpack6 rev2 on my devkit with an NVME.
Follow the steps, I’m so happy my VLM server starts and show “ready”
But during the download, I observe that the network activity get stop?
And finally it gets OK?

now I can access the dashboard http://[NVStreamer-IP]:31000
but for http://[JETSON-IP]:30080/vst/
it shows “Unable to connect”
What can I do?
5015

30080
300801829×1057 57.4 KB

31000
310001829×1057 138 KB

9

Are you copy below file based on VLM guide: Visual Language Models (VLM) with Jetson Platform Services — Metropolis on Jetson documentation 0.1.0 documentation ?

sudo cp config/vlm-nginx.conf /opt/nvidia/jetson/services/ingress/config
sudo cp config/prometheus.yml /opt/nvidia/jetson/services/monitoring/config/prometheus.yml
sudo cp config/rules.yml /opt/nvidia/jetson/services/monitoring/config/rules.yml

10

I’m sure I did these 3 statements.
And I did that once again and restarted the 4 services (sudo systemctl start jetson-ingress …)
But it cannot wok neither.

11

After several reboots and restart vst
It works now.
Thanks