Get started quickly with the NIM framework, and an error occurred when trying to reproduce it

Accelerated Computing NGC GPU Cloud
1

Hi, guys

I followed the operation steps in the blog (https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/) for reproduction, but encountered the following error.

root@a100-test-ser000:~$ ngc registry model download-version “ohlfw0olaadg/ea-participants/llama-2-7b:LLAMA-2-7B-4K-FP16-1-A100.24.01”
Client Error: 403 Response: Access Denied - Request Id: ad9d65d2-4328730 Url: https://api.ngc.nvidia.com/v2/org/ohlfw0olaadg/team/ea-participants/models/llama-2-7b/versions/LLAMA-2-7B-4K-FP16-1-A100.24.01

root@a100-test-ser000:~$ ngc registry model list “ohlfw0olaadg/ea-participants/llama-2-7b:*”
‘NoneType’ object is not iterable

Is this repository a private repository that developers do not have permission to access?

If we want to reproduce the functionality of NIM, is there a more detailed quick start tutorial?

Thank you.

1 Like
2

I would like to do the same. Also, obtained some more details with --debug on the model list command

Traceback (most recent call last):
  File "ngccli/ngc.py", line 156, in main
  File "ngcbpc/tracing.py", line 174, in wrapper_span
  File "ngcbpc/command/clicommand.py", line 216, in traced_func
  File "registry/command/model.py", line 193, in list
TypeError: 'NoneType' object is not iterable
1 Like
3

Can any bloggers help answer this question? Thanks.

@ Amanda Saunders @ Neal Vaidya @ Adam Tetelman @ Nik Spirin

5

I am also facing the same issue can any one answer how to deploy llm with NIM

Client Error: 403 Response: Access Denied - Request Id: 3919aa59-213897 Url: https://api.ngc.nvidia.com/v2/org/ohlfw0olaadg/team/ea-participants/models/llama-2-7b/versions/LLAMA-2-7B-4K-FP16-1-A100.24.01

6

To get started with a NIM do I need an Ingress/Egress controller and what’s recommended?

7

To access NIM prerequisites, you need to apply for early access authorization.
You need to find NV’s enterprise-level sales.
You can refer to the following Chinese community posts. https://zhuanlan.zhihu.com/p/684106121

8

NVIDIA Microservices is currently under limited availability. Please submit a request to get access.

image
image1920×933 91.8 KB