Issue detecting video in NVIDIA VSS

[taopikhidayat27]

Hello, Support team.
I’ve already run NVIDIA VSS and tested it with a sample traffic video. But in both the summary and Q&A, it’s not detecting any collisions or accidents. Could you help me figure this out?

Sincerely
Taopik H.

[yuweiw]

Sorry for the late reply. We will use the video you provided and try that on our side first.

[taopikhidayat27]

Thank you very much.
I’ve already rented 8 * A100 (80 GB) GPUs and deployed everything using Helm.
When I deployed, I used an overrides.yaml file like this:

---

nim-llm:
env:

• name: NVIDIA_VISIBLE_DEVICES
  value: “0,1,2,3”
  resources:
  limits:
  nvidia.com/gpu: 0 # no limit

vss:
applicationSpecs:
vss-deployment:
containers:
vss:
env:
- name: VLM_MODEL_TO_USE
value: nvila
- name: MODEL_PATH
value: “git:Efficient-Large-Model/NVILA-15B · Hugging Face”
- name: NVIDIA_VISIBLE_DEVICES
value: “4,5”
resources:
limits:
nvidia.com/gpu: 0 # no limit

nemo-embedding:
applicationSpecs:
embedding-deployment:
containers:
embedding-container:
env:
- name: NGC_API_KEY
valueFrom:
secretKeyRef:
key: NGC_API_KEY
name: ngc-api-key-secret
- name: NVIDIA_VISIBLE_DEVICES
value: ‘6’
resources:
limits:
nvidia.com/gpu: 0 # no limit

nemo-rerank:
applicationSpecs:
ranking-deployment:
containers:
ranking-container:
env:
- name: NGC_API_KEY
valueFrom:
secretKeyRef:
key: NGC_API_KEY
name: ngc-api-key-secret
- name: NVIDIA_VISIBLE_DEVICES
value: ‘7’
resources:
limits:
nvidia.com/gpu: 0 # no limit

---

I used the NVILA-15B model, but when I test it, it doesn’t detect any accidents accurately.
This is a problem. Please help me with this.

Sincerely,
Taopik H.

[yuweiw]

I have tried that with the highres nvila model on my side. It can detect the accident. Could you try that?

          - name: MODEL_PATH
            value: "ngc:nvidia/tao/nvila-highres:nvila-lite-15b-highres-lita"

[taopikhidayat27]

Could you please provide me with the detailed overrides.yaml file? Also, let me know which GPU(s) you rented.

And according to the documentation, I can only see the alert API for live streaming. I also need alert APIs for both video and picture summarization. Could you assist with this as well?

Sincerely,
Taopik H

[yuweiw]

@taopikhidayat27

Could you please provide me with the detailed overrides.yaml file?

override.yaml (1.4 KB)

@taopikhidayat27

Could you assist with this as well?

You can create alerts for any sources.

[taopikhidayat27]

Thanks for providing detail scripts.

But regarding alert api, what I want to know is when I summarize video or image, how can I get alert result if I use which apis? could you please detail guide for getting alert using api?

I attached alert and summarization apis. But I am not clear how can get alert result.
Please help me.

Sincerely
Taopik H.

[taopikhidayat27]

Hello,
I have deployed highres nvila model.
Summary shows accident but Q&A doesnt show accidents. And It is not Generating Video Highlight.

Could you help this problem?

Sincerely,
Taopik H.

[taopikhidayat27]

Hello,
Please help me with detecting accidents on Q&A.

Sincerely,
Taopik H.

[yuweiw]

I have tried that on my side.

1. click the button below
   

   

2. input the “accident” in the input field.

The highlight clips can be generated normally.

[taopikhidayat27]

Thanks for responding.

I have deployed highres nvila model.
Summary shows accident but Q&A sometimes doesnt show accidents. It is inconsistent.
Could you help this problem?

And is it possible to deploy VILA1.5-40B model not highres in 8 * A100 GPU?

[yuweiw]

I have tried that on my side. It works normally. Could you try to delete the video and summarize that again?