Topics tagged inference-server-triton

Topic	Replies	Views	Activity
/generate silently drops forwarded HTTP headers — we're flying blind on multi-tenant LLM traffic (PR #8916 open) JAX inference-server-triton	0	32	August 24, 2026
Infer vs generate for Triton ensemble models cuDNN inference-server-triton	0	39	August 9, 2026
Ckg-nvidia-ai — NVIDIA AI stack as a traversable MCP knowledge graph (NIM, NeMo, AgentIQ, Isaac, 20 domains) Models nemo , ai , inference-server-triton , cublas , nim , agentic-ai , cosmos , nemo-agent-toolkit	3	243	July 9, 2026
Encountering 0 bytes input in asynchronous internal model call in Business Logic Scripting even though image is non-empty Nemotron Speech + Riva inference-server-triton	2	104	June 30, 2026
Triton Inference Server Support Matrix lists incorrect PyTorch version for release 26.05 Docker and NVIDIA Docker tensorrt , docker , cudnn , inference-server-triton , cublas , docker-machine-learning	0	132	June 20, 2026
From Governance Runtime Assurance to Human-Directed Intelligence Confidential Computing python , ai , inference-server-triton , architecture-and-design , system-management-and-architecture , nim , llm , agentic-ai	7	216	June 19, 2026
From Deterministic Inference to Governance Runtime Assurance — Version 2 Control-Plane Architecture Base Command Manager pytorch , python , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim , agentic-ai	0	112	June 2, 2026
Governance Runtime Assurance — Measuring Route Reliability Beyond Raw Inference Speed Base Command Manager pytorch , python , ai-training , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim , agentic-ai	0	69	May 28, 2026
Triton inference on multi GPU has slow inference with incorrect results GPU-Accelerated Libraries cuda , inference-server-triton , linux-driver	2	122	May 26, 2026
Live Orchestration Intelligence — Persistent Route Memory for Governance-Native AI Factory Control Planes Fleet Intelligence cuda , kernel , pytorch , python , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim	0	100	May 20, 2026
Runtime Optimization vs Governance Runtime Engineering — Parallel Acceleration Above the Model Layer Base Command Manager tensorrt , cuda , inference-server-triton , artificialintelligence , system-management-and-architecture , nim	0	81	May 16, 2026
DeepStream 8.0 SCRFD + ArcFace: How to Pass Facial Landmark Metadata for Warp Affine Before SGIE? DeepStream SDK inference-server-triton , deepstream	5	165	May 14, 2026
Runtime Optimization vs Governance Orchestration — A New AI Acceleration Layer Emerging Above the Model Base Command Manager tensorrt , cuda , inference-server-triton , artificialintelligence , nim , humanoid-robotics	0	119	May 11, 2026
Experiences running Qwen/Qwen3-Coder-Next? Jetson Thor inference-server-triton , generative_ai	10	1878	March 16, 2026
Optimize .NET Real-Time Video Pipeline with Multiple TensorRT Models — Low GPU Utilization & Throughput Bottleneck Computer Vision & Image Processing tensorrt , inference-server-triton	0	84	February 2, 2026
tritonclient.utils.InferenceServerException: Fail to connect to remote host ipv4:127.0.0.1:8001 in TRELLIS NIM Models inference-server-triton , nim	1	268	December 19, 2025
CUDA Buffer Sharing Failure Between Triton and DeepStream Containers on WSL2 DeepStream SDK cuda , nvbugs , wsl , inference-server-triton , deepstream	6	215	December 17, 2025
Deterministic Inference at Scale: Moving Beyond Agents and MoE in Regulated Workloads TensorRT jetson-inference , inference-server-triton , nim , llama	2	347	December 15, 2025
TensorRT built-in NMS output lost when using Triton dynamic batching TensorRT tensorrt , cudnn , inference-server-triton	2	286	December 2, 2025
Bug Report Summary \| Product : NVIDIA NIM for Image OCR (NeMo Retriever OCR v1) \| Version: 1.1.0 \| Severity: High (Production Blocker) Technical Support (PhysicsNeMo Only) nemo , inference-server-triton , gpu , nim , deepseek	0	160	November 18, 2025
Segmentation Fault Loading YOLO v4 TensorRT Model with Triton TensorRT tensorrt , inference-server-triton	1	153	November 18, 2025
NIM to Triton Server Pipeline Models inference-server-triton , nim	1	224	November 14, 2025
Creating a container for seminar Fundamentals of Deep Learning Courses and Workshops cuda , kernel , docker , pytorch , inference-server-triton	0	53	November 9, 2025
Nvinfer yields constant OCR text with NHWC engine (fast_plate_ocr – cct_s_v1_global_model) while nvinferserver returns correct results DeepStream SDK inference-server-triton , deepstream	2	155	November 7, 2025
Gray image in Triton DeepStream SDK gstreamer , inference-server-triton , deepstream	2	117	October 31, 2025
Running Llama-3.1-8B-FP4 get triton error. Value 'sm_121a' is not defined for option 'gpu-name' DGX Spark / GB10 inference-server-triton , llama	2	790	October 24, 2025
Tensor-RT rejects engine cache pre-built on same device type Jetson AGX Orin tensorrt , cudnn , inference-server-triton	4	280	October 2, 2025
Connection problem due to lack of CORS support in Triton Server, which blocks requests from frontend web applications TensorRT python , cudnn , onnx , inference-server-triton	3	235	September 12, 2025
Triton + TensorRT-LLM (Llama 3.1 8B) – Feasibility of Stateful Serving + KV Cache Reuse + Priority Caching JAX inference-server-triton , llama	1	246	September 5, 2025
How to access labelfile_path in custom classifier parser for nvinferserver? DeepStream SDK gstreamer , inference-server-triton , jetson , deepstream	2	136	August 19, 2025

/generate silently drops forwarded HTTP headers — we're flying blind on multi-tenant LLM traffic (PR #8916 open)

JAX

inference-server-triton

0

32

August 24, 2026

Infer vs generate for Triton ensemble models

cuDNN

inference-server-triton

0

39

August 9, 2026

Ckg-nvidia-ai — NVIDIA AI stack as a traversable MCP knowledge graph (NIM, NeMo, AgentIQ, Isaac, 20 domains)

Models

nemo , ai , inference-server-triton , cublas , nim , agentic-ai , cosmos , nemo-agent-toolkit

3

243

July 9, 2026

Encountering 0 bytes input in asynchronous internal model call in Business Logic Scripting even though image is non-empty

Nemotron Speech + Riva

inference-server-triton

2

104

June 30, 2026

Triton Inference Server Support Matrix lists incorrect PyTorch version for release 26.05

Docker and NVIDIA Docker

tensorrt , docker , cudnn , inference-server-triton , cublas , docker-machine-learning

0

132

June 20, 2026

From Governance Runtime Assurance to Human-Directed Intelligence

Confidential Computing

python , ai , inference-server-triton , architecture-and-design , system-management-and-architecture , nim , llm , agentic-ai

7

216

June 19, 2026

From Deterministic Inference to Governance Runtime Assurance — Version 2 Control-Plane Architecture

Base Command Manager

pytorch , python , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim , agentic-ai

0

112

June 2, 2026

Governance Runtime Assurance — Measuring Route Reliability Beyond Raw Inference Speed

Base Command Manager

pytorch , python , ai-training , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim , agentic-ai

0

69

May 28, 2026

Triton inference on multi GPU has slow inference with incorrect results

GPU-Accelerated Libraries

cuda , inference-server-triton , linux-driver

2

122

May 26, 2026

Live Orchestration Intelligence — Persistent Route Memory for Governance-Native AI Factory Control Planes

Fleet Intelligence

cuda , kernel , pytorch , python , inference-server-triton , artificialintelligence , architecture-and-design , system-management-and-architecture , nim

0

100

May 20, 2026

Runtime Optimization vs Governance Runtime Engineering — Parallel Acceleration Above the Model Layer

Base Command Manager

tensorrt , cuda , inference-server-triton , artificialintelligence , system-management-and-architecture , nim

0

81

May 16, 2026

DeepStream 8.0 SCRFD + ArcFace: How to Pass Facial Landmark Metadata for Warp Affine Before SGIE?

DeepStream SDK

inference-server-triton , deepstream