Topics tagged inference-server-triton

Topic	Replies	Views	Activity
Is it possible to deploy the Llama-70b model with TensorRT LLM on an L40S GPU? TensorRT tensorrt , ubuntu , inference-server-triton	1	65	May 9, 2024
Triton infer server docker Orin NX 5.1.1 fails to start DeepStream SDK docker , inference-server-triton , jetson	1	61	May 9, 2024
Cannot use model-analyzer on ONNX classification model with dynamic input DeepStream SDK performance , inference-server-triton , performance-metrics	1	70	May 6, 2024
Triton Error: UNAVAILABLE: Invalid argument: unable to load model 'pose_classifier_tensorrt', configuration expects 2 inputs, model provides 1 DeepStream SDK inference-server-triton	1	103	May 3, 2024
Not able to to recover the video/channels using new streammux plus triton inference server General Topics and Other SDKs inference-server-triton , deepstream	1	152	May 2, 2024
Is it posible to run Triton Server on GPU device and Gstreamer with Nvinterserver in a CPU-Only device? DeepStream SDK gstreamer , inference-server-triton , grpc	4	145	May 12, 2024
Order within triton inference server python backend DeepStream SDK python , inference-server-triton , deepstream	31	644	May 6, 2024
Avoid memory copy for deepstream pipeline connecting to a standalone local triton inference server DeepStream SDK docker , inference-server-triton , gpu , grpc , deepstream	2	191	April 1, 2024
How to correctly format data on the client side to send to dali/triton AI Foundation Models and Endpoints inference-server-triton	0	104	April 14, 2024
Unable to run Triton example TensorRT inference-server-triton	0	206	April 9, 2024
Help with Nvidia Triton Inference Server Installation: TensorRT 8.6.3 Version Unavailable TensorRT inference-server-triton	0	109	April 8, 2024
TTS Synthesize Online randomly fails with a Streaming timed out Riva inference-server-triton , riva	1	272	April 5, 2024
Has anyone gotten working speaker diarization on triton? Specifically with the Multiscale Diarization Decoder (Diarization MSDD) or Neural Diarizer) AI Foundation Models and Endpoints nemo , inference-server-triton	0	112	April 3, 2024
Facing failed to load 'yolo' version 1: Internal: onnx runtime error 1: Load model from /data/yolo/1/best.onnx failed:Fatal error: TRT:EfficientNMS_T TensorRT onnx , inference-server-triton	0	120	April 3, 2024
Installing Triton Server on Lenovo SE70 with Xavier NX Jetson Xavier NX inference-server-triton	20	590	April 22, 2024
Cannot start triton server (command returned a non-zero code: 126) DeepStream SDK inference-server-triton	3	318	March 28, 2024
Help with efficient execution of triton ensembles DeepStream SDK inference-server-triton	8	272	March 1, 2024
Nvinferserver apps crashing just by importing torch DeepStream SDK inference-server-triton	8	471	February 22, 2024
Triton Inference Server, Model Analyzer Triton Inference Server - archived inference-server-triton	0	165	March 4, 2024
Unable to load yolov7 model into triton inference server on Jetson Orin Developer kit Jetson AGX Orin inference-server-triton	7	240	March 12, 2024
Triton infirence GPU-Accelerated Libraries inference-server-triton	0	160	February 23, 2024
Triton server getting error JAX inference-server-triton	0	218	February 14, 2024
Performance data mistakes in LLAMA inference CUDA Programming and Performance tensorrt , natural-language-processing-nlp , inference-server-triton	1	236	February 7, 2024
Issues with using multiple perf-analyzer processes for Triton Inference Server Triton Inference Server - archived performance , inference-server-triton	0	358	February 5, 2024
Inference speed of Triton Server Triton Inference Server - archived tensorrt , python , inference-server-triton	0	410	December 19, 2023
Triton with python backend crashes when running on multi-gpu server GPU - Hardware inference-server-triton	0	486	December 22, 2023
Xavier NX restarts while running AI models Jetson Xavier NX tensorrt , cudnn , inference-server-triton	2	275	December 19, 2023
Xavier NX restarts while running AI models TensorRT tensorrt , cudnn , inference-server-triton	2	273	December 18, 2023
Can the python backend of TIS be used to serve larger models? Frameworks inference-server-triton	0	199	December 14, 2023
Deepstream yolov8 trition server load the model plan DeepStream SDK inference-server-triton , deepstream61	4	437	December 8, 2023