Language |
Mistral 7B Instruct |
IGI Plugin |
ONNX |
DML |
Intel, AMD, NVIDIA GPUs or CPUs |
|
|
NIM |
TensorRT |
TensorRT-LLM |
Cloud, On-prem or Local GPUs |
Language |
Mistral Nemo Family of Models (2B, 4B, 8B) |
IGI Plugin |
GGUF |
GGML |
Intel, AMD, NVIDIA GPUs or CPUs |
|
|
NIM |
.nemo checkpoint |
TensorRT-LLM |
Cloud, On-prem or Local GPUs |
Vision |
Nemovision 4B Instruct V2 |
IGI Plugin |
GGUF |
GGML |
Intel, AMD, NVIDIA GPUs or CPUs |
Memory |
E5 Large Unsupervised Embedded RAG |
IGI Plugin |
GGUF |
GGML |
Intel, AMD, NVIDIA GPUs or CPUs |
Speech |
Riva Magpie Flow TTS |
IGI Plugin |
ONNX |
DML + TRT |
Intel, AMD, NVIDIA GPUs or CPUs |
Speech |
Whisper ASR |
IGI Plugin |
GGUF |
GGML |
Intel, AMD, NVIDIA GPUs or CPUs |
|
|
NIM |
ONNX |
Triton + TRT |
Cloud, On-prem or Local GPUs |
Speech |
Riva ASR |
IGI Plugin |
TRT |
Triton + TRT |
NVIDIA GPUs |
|
|
NIM |
RMIR |
Triton + TRT |
Cloud, On-prem or Local GPUs |