GPU status before starting VSS.
Nemo embedding keeps CrashLoopBackOff and failed to start vss deployment.
I attach the full log of nemo embedding(which keeps CrashLoopBackOff) and vss-deployment.
I see the error log in vss-deployment that says
GuardRails model load execution time = 2.563 sec
2025-02-11 01:58:40,941 ERROR Failed to load VIA stream handler - Guardrails failed
After applying the override file(force GPU allocation), like the image below(pod status), nemo embedding and vss-deployment keeps restarting.
Log is same as above.
tnx :)
vss_deployment_log.txt (148.5 KB)
nemo_emb_log.txt (33.8 KB)