Hey @rawnak.kumar, I saw your other post in the VIA forum – have you looked at the “Serving models from Local Assets” section of this page? It might help with your usecase.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Unable to Run NIM on H100 GPU Due to Profile Compatibility Issue Despite Sufficient GPU Resources | 1 | 49 | November 12, 2024 | |
| Getting Started With NVIDIA NIM Tutorial Issues with NGC Registry | 7 | 530 | July 24, 2024 | |
| Nemollm-inference-microservice failed to deploy | 1 | 60 | October 22, 2024 | |
| NIM does not support llama-3.1-8b-instruct and llama-3.1-70b-instruct on GH200 On-Prem deployment | 1 | 57 | November 7, 2024 | |
| Get started quickly with the NIM framework, and an error occurred when trying to reproduce it | 6 | 921 | April 26, 2024 | |
| Issues while starting NIM container in A10 VM | 4 | 79 | September 4, 2024 | |
| NIM with llama-3-8b models stuck without any error | 0 | 37 | November 15, 2024 | |
| NIM nim/meta/llama3-8b-instruct - no API key is detected | 2 | 494 | July 23, 2024 | |
| NIM authentication issues | 4 | 393 | July 18, 2024 | |
| How to fix 0 compatible profiles? Where to get compatible profiles? | 4 | 223 | November 26, 2024 |