Finding the Kubernetes manifest for any NIM model

AI & Data Science NVIDIA NIM Models
, ,
1

In NVIDIA’s page they have the Kubernetes manifests only in some of the models
The rest they have usually docker deployment commands.

Is there any way to find the complete CRD kind: NIMService for any available model in NVIDIA?

An example is the below

apiVersion: apps.nvidia.com/v1alpha1
kind: NIMService
metadata:
name: llama-32-nv-embedqa-1b-v2
namespace: nim-service
spec:
image:
repository: nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2
tag: latest
pullPolicy: IfNotPresent
pullSecrets:
- ngc-secret
authSecret: ngc-api-secret
storage:
pvc:
create: true
size: “model-size”
volumeAccessMode: “ReadWriteOnce”
replicas: 1
resources:
limits:
nvidia.com/gpu: 1
expose:
service:
type: ClusterIP
port: 8000

Do we use always the same manifest/CRD for all NIMs but we just change the image?
Or there is place where I can find the exact Kubernetes manifest for each model (LLM/EMbeddings etc)?

2

Hey @TonyDim – you can check out the NIM Operator docs for more details but yeah essentially you just change the the image and potentially the number of GPUs for different NIM