Launch the Reranker NIM : Failing to create container for

[paul368]

When running in Jupyter or a Brev CLI :
!sudo docker run -it --rm
–gpus ‘“device=4”’
–shm-size=16GB
-e NGC_API_KEY
-v “$LOCAL_NIM_CACHE:/opt/nim/.cache”
-u $(id -u)
-p 9235:8000
nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:latest

I get the error:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Using requested mode ‘cdi’
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime (e.g. specify the --runtime=nvidia flag) instead.: unknown.

[aryason]

Hello Paul,
What instance type did you provision (GPUs and CSP)?

[paul368]

Hi Adam

This is the instance I have :

Thanks
Paul

[shuagrawal]

Did you modify the command to launch this NIM? For example, it should have two ‘-’ before gpus: “–gpus”

what error do you get when you run:

!docker run -it --rm
–gpus ‘“device=4”’
–shm-size=16GB
-e NGC_API_KEY
-v “$LOCAL_NIM_CACHE:/opt/nim/.cache”
-u $(id -u)
-p 9235:8000
-d
nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:latest

[paul368]

Thank you.

I now get this error :

Unable to find image '[nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:latest](http://nvcr.io/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2:latest)' locally
docker: Error response from daemon: Head "[https://nvcr.io/v2/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2/manifests/latest](https://nvcr.io/v2/nim/nvidia/llama-3.2-nv-rerankqa-1b-v2/manifests/latest)": unknown: {"errors": [{"code": "DENIED", "message": "Payment Required"}]}.
See 'docker run --help'.

But I have credit :

Can I dig into that error further ? WHere does it expect to lookup credit ? 

[yuweiw]

Could you check your security groups by referring to the #9 first?

[louis50]

Getting this error as well and I have credits. Any solution to this?

[yuweiw]

You can refer to this #9, there are no more credits associated with accounts. Could you try the latest version of VSS?