Topics tagged llama-31-8b-instruct

Topic	Replies	Views	Activity
Trying to Run DAPT-Continual PreTraining on Chip_design Data NVIDIA NeMo cuda , nemo , llama-31-8b-instruct , llama	2	31	August 5, 2025
NVIDIA NIM: LLAMA-4 (Maverick) Image for performance Benchmarking on Nvidia H200 GPUs Models cuda , nim , llama-31-8b-instruct , llama	0	76	July 20, 2025
Speculative decoding using vLLM on the Nvidia Jetson AGX Orin 64GB dev kit Jetson Projects generative_ai , llama-31-8b-instruct , llama	0	170	March 9, 2025
VSS local deployment single gpu: Failed to load VIA stream handler - Guardrails / CA-RAG setup failed Visual AI Agent nim , llama-31-8b-instruct , llama	4	96	July 4, 2025
Example-hybrid-rag NVIDIA AI Workbench nim , llama-31-8b-instruct , llama	7	112	June 2, 2025
Model _ request Model Does not exist error NIM on RTX AI PCs and Workstations nim , llama-31-8b-instruct , llama	0	35	May 31, 2025
SOTA inference speed using SGlang and EAGLE-3 speculative decoding on the NVIDIA Jetson AGX Orin Jetson Projects llama-31-8b-instruct , llama	2	640	March 23, 2025
Batch processing using NVIDIA NIM \| Docker \| Self-hosted Models python , nim , llama3-8b-instruct , llama-31-8b-instruct , llama	11	443	January 29, 2025
Running LMdeploy inference engine on the NVIDIA Jetson AGX Orin Devkit Jetson Projects jetson , llama-31-8b-instruct , llama	2	156	January 14, 2025
Failed to MLC-compile mlc-ai/Llama-3.1-8B-Instruct-fp8-MLC on Jetson AGX orin Jetson AGX Orin generative_ai , llama-31-8b-instruct , llama	5	177	January 13, 2025
Jetson Orin Nano Super Dev Kit Performance Jetson Orin Nano cudnn , gemma-2-9b-it , llama-31-8b-instruct , llama	6	903	January 28, 2025
How to fix 0 compatible profiles? Where to get compatible profiles? Models nim , llama-31-8b-instruct , llama	4	560	November 26, 2024
Boosting LLM Inference Speed Using Speculative Decoding in MLC-LLM on Nvidia Jetson AGX Orin Jetson Projects generative_ai , llama-31-8b-instruct , llama	0	227	November 23, 2024
NIM TensorRT-LLM on H100 NVL Models nim , llama-31-8b-instruct , llama	2	179	November 22, 2024
Unable to Run NIM on H100 GPU Due to Profile Compatibility Issue Despite Sufficient GPU Resources Models nim , llama-31-8b-instruct , llama	1	235	November 12, 2024
NIM does not support llama-3.1-8b-instruct and llama-3.1-70b-instruct on GH200 On-Prem deployment Models nim , llama-31-8b-instruct , llama	1	256	November 7, 2024
Reusing a stored model (llama-3.1-8b-instruct) with a proper profile Models nim , llama-31-8b-instruct , llama	0	173	October 30, 2024
LoRA swapping inference Llama-3.1-8b-instruct \| Exception: lora format could not be determined Models nim , llama3-8b-instruct , llama-31-8b-instruct , llama	4	188	October 22, 2024
Running Ollama / llama3.1 on Jetson AGX Xavier 16gb is it possible? how-to? Jetson AGX Xavier generative_ai , llama-31-8b-instruct	8	2322	October 19, 2024
NIM API key not Found Models nim , llama-31-8b-instruct , llama	4	701	September 21, 2024
API connect Models nim , llama-31-8b-instruct , llama	1	176	September 20, 2024
Problem with installation of Llama 3.1 8b NIM Models nim , llama3-8b-instruct , llama-31-8b-instruct , llama	1	584	September 4, 2024
Fail to evaluate LLM efficiency using nemo evaluator General cuda , docker , python , inference-server-triton , kubernetes , nim , llama-31-8b-instruct , llama	0	20	August 28, 2024
OpenAI Compatible API does not work Models llama-31-8b-instruct , llama-31-70b-instruct	6	511	August 26, 2024
Llama3.1-8b error on startup in k8s Models llama-31-8b-instruct	6	305	August 13, 2024