A Simple Guide to Deploying Generative AI with NVIDIA NIM

Originally published at: A Simple Guide to Deploying Generative AI with NVIDIA NIM | NVIDIA Technical Blog

Whether you’re working on-premises or in the cloud, NVIDIA NIM inference microservices provide enterprise developers with easy-to-deploy optimized AI models from the community, partners, and NVIDIA. Part of NVIDIA AI Enterprise, NIM offers a secure, streamlined path forward to iterate quickly and build innovations for world-class generative AI solutions. Using a single optimized container, you…

Hello! I am trying to carefully follow the Getting started NIM blog above to run the NIM image standalone on my GPU based AWS EC2 instance but I’m blocked by this strange docker authentication issue:

docker login nvcr.io
Authenticating with existing credentials… WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. Configure a credential helper to remove this warning.
$ docker pull nvcr.io/nim/meta/llama3-8b-instruct:24.05
Error response from daemon: Head “https://nvcr.io/v2/nim/meta/llama3-8b-instruct/manifests/24.05”: unauthorized: 401 Authorization Required

401 Authorization Required


nginx/1.22.1

I am using the export NGC_API_KEY= for authentication.

Any suggestions on why direct authentication into ‘nvcr.io’ using my key works:
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
Configure a credential helper to remove this warning. See
Login Succeeded`

but this command just trying to pull down an image from nvcr.io:
`docker pull nvcr.io/nim/meta/llama3-8b-instruct:24.05
Error response from daemon: Head "XXXXXX : unauthorized:

401 Authorization Required

401 Authorization Required


nginx/1.22.1

What am I missing please? thanks!

Note that an NVIDIA AI Enterprise License is required to download and use NIM.

1 Like

Any guide to apply NVIDIA AI Enterprise License? Thanks