Error running NVIDIA VSS || pods keep restarting and crashing multiple times

[taopikhidayat27]

I faces with some issues while deploying the helm chart, I build the Kubernetes pods, they kept restating and crashing multiple times and never completed

I have rent the following GPU:
8 x A100 (80 GB)
RAM: 256GB
CUDA version 12.2
Nvidia Driver : 535.161.08

According to official NVIDIA VSS doc(Setup the Prerequisites — Video Search and Summarization Agent), I successfully installed Kubernetes Cluster, Create Required Secrets, Fetch and install the VSS Blueprint Helm Chart.
But when I run the following command-“sudo watch microk8s kubectl get pod”
vss-vss-deployment-xxxxxx crashed repeatedly, thus I can’t lunch VSS.
Please help this problem.

[yuweiw]

Please refer to our FAQ to get more information.

[taopikhidayat27]

I have already reviewed FAQ. But I can’t find the solution.
Only “vss-vss-deployment-xxxx” doesn’t run and continue to restart. Could you tell me the reason?

[taopikhidayat27]

hi, yuweiw. Please give me guide

[yuweiw]

Please click the link in my last comment, and get the log according to that FAQ, then we can analyze it further.

[taopikhidayat27]

Hello, Still doesn’t work. vss-vss-deployment-xxxx continue CrashLoopBackOff.
By the way, my purpose is to get detail VSS API Doc. To get this, do I have to deploy surely NVIDIA VSS? Could you guide me to get detail API doc?

Thanks.

[taopikhidayat27]

Hello, yuweiw.
I get the log according to the FAQ. I attached screen and file.

sudo microk8s kubectl logs vss-vss-deployment-POD-NAME
sudo microk8s kubectl logs vss-vss-deployment-POD-NAME.txt (6.0 KB)

sudo microk8s kubectl describe pod vss-vss-deployment-POD-NAME
sudo microk8s kubectl describe pod vss-vss-deployment-POD-NAME.txt (11.4 KB)

sudo microk8s.kubectl exec -it vss-vss-deployment-POD-NAME – /bin/bash

**

Please check this and give me guide.

Sincerely
Taopik Hidayat.

[yuweiw]

OK. Your problem may be similar to this #7.
You can try replacing the VILA to NVILA with the -f parameter.

sudo microk8s helm install vss-blueprint nvidia-blueprint-vss-2.2.0.tgz --set global.ngcImagePullSecretName=ngc-docker-reg-secret -f override.yaml

override.yaml (1.4 KB)

[taopikhidayat27]

Thanks yuweiw. I successfully deployed. But how can I access the VSS in my local?
Please refer this files.

Sincerely
Taopik Hidayat.

[taopikhidayat27]

Thanks yuweiw. I successfully deploy VSS and get detail API doc. Thanks you again with your support!