TensorRT

Topic	Replies	Views	Activity
TensorRT 8.2 and Framework Integrations are out now! tensorrt The latest release of TensorRT, 8.2 GA, and TensorRT Integrations for PyTorch and TensorFlow, is now available for download. Highlight includes: TensorRT 8.2 Optimizations for T5 and GPT-2 deliver real time translati…	5	5841	November 6, 2023
Best practices for reporting an issue or bug relating to TensorRT	0	8776	September 21, 2018
Inference VILA 3b using The Triton TensorRT-LLM Backend	0	3	December 22, 2024
MPI error after loading TensorRT engines on Triton cudnn , inference-server-triton , gemma-2-9b-it	0	15	December 19, 2024
Trt10.5/10.7 trtexec convert onnx model failed （Error Code 2: Internal Error (Assertion !mValueMapUndo failed. )） tensorrt , cudnn	4	23	December 18, 2024
Cuda Runtime (invalid resource handle) when use TensorRT and Pytorch(on GPU) simultaneously	5	2817	December 17, 2024
Unable to create tensorRT engine file for yolov8s-seg model in ds7.1 deepstream	5	30	December 17, 2024
Hello, may I ask why this error occurred when converting the model tensorrt , cudnn , jetson	1	18	December 17, 2024
TensorRT-10.5.0.18 nvinfer_10.dll possibly corrupted or not fully Windows made? tensorrt	3	85	December 16, 2024
Issue with TensorRT Whisper consuming inordinate amount of system memory which kills the process tensorrt , cuda , pytorch , cudnn	2	444	December 14, 2024
Reducing inference time for a custom model	0	21	December 13, 2024
When I use TensorRT enqueueV2 it doesn't work with no error tensorrt , cuda	0	15	December 12, 2024
How TensorRT IOPlugin get enough workspace size in explicit batch mode? tensorrt	0	8	December 12, 2024
OCRnet Resnet 50 issue while deploying with custom character list tensorrt , opencv , cuda , gstreamer , inference-server-triton	0	6	December 12, 2024
Can multiple CUDA contexts share an inference engine? tensorrt , cuda	0	9	December 11, 2024
TensorRT Error: `[genericReformat.cuh::copyVectorizedRunKernel::1579] Error Code 1: Cuda Runtime (invalid resource handle)` in ROS Callback Function tensorrt , ros , cuda , cudnn	0	15	December 10, 2024
Yolov11 Triton Inference Server Deployment Problem tensorrt , inference-server-triton	0	24	December 7, 2024
Could not find any implementation for node cudnn	21	163	December 6, 2024
Could not find any implementation for node failure of TensorRT 8.5 when running on GPU Jetson Xavier NX	0	11	December 6, 2024
Best websites or books on TensorRT optimization	2	23	December 5, 2024
Sparsity on Onnx Model	0	14	December 4, 2024
Int8 implementation of element-wise ops with multiple inputs	2	21	December 2, 2024
gRPC end point unavailable status on triton inference server python , cudnn , inference-server-triton , grpc	0	7	December 2, 2024
Error adding symbols. Tensorrt installation cudnn	2	27	December 2, 2024
Clarification on TensorRT Binding Order and Guarantee of Input-Output Sequence (bindingIsInput() etc) tensorrt	1	28	December 2, 2024
How can I optimize multi-batch and parallel inference in TensorRT for faster performance on high-resolution image patches? tensorrt , cuda , ubuntu , python , cudnn , deep-learning	2	56	December 2, 2024
Which version of pytorch and python is compatible with cuda 11.4 cuda , pytorch , python	0	36	November 30, 2024
Tensorrt-llm Phi-3-mini-128k-instruct error cudnn , phi-3-mini-128k-instruct	1	44	November 30, 2024
Tensorrt Conversion	2	50	November 30, 2024
Error while deploying the Speech and NLP models for ACE	1	23	November 30, 2024

TensorRT 8.2 and Framework Integrations are out now!

tensorrt

The latest release of TensorRT, 8.2 GA, and TensorRT Integrations for PyTorch and TensorFlow, is now available for download. Highlight includes: TensorRT 8.2 Optimizations for T5 and GPT-2 deliver real time translati…

5

5841

November 6, 2023

Best practices for reporting an issue or bug relating to TensorRT

0

8776

September 21, 2018

Inference VILA 3b using The Triton TensorRT-LLM Backend

0

3

December 22, 2024

MPI error after loading TensorRT engines on Triton

cudnn , inference-server-triton , gemma-2-9b-it

0

15

December 19, 2024

Trt10.5/10.7 trtexec convert onnx model failed （Error Code 2: Internal Error (Assertion !mValueMapUndo failed. )）

tensorrt , cudnn

4

23

December 18, 2024

Cuda Runtime (invalid resource handle) when use TensorRT and Pytorch(on GPU) simultaneously

5

2817

December 17, 2024

Unable to create tensorRT engine file for yolov8s-seg model in ds7.1

deepstream

5

30

December 17, 2024

Hello, may I ask why this error occurred when converting the model

tensorrt , cudnn , jetson

1

18

December 17, 2024

TensorRT-10.5.0.18 nvinfer_10.dll possibly corrupted or not fully Windows made?

tensorrt

3

85

December 16, 2024

Issue with TensorRT Whisper consuming inordinate amount of system memory which kills the process

tensorrt , cuda , pytorch , cudnn

2

444

December 14, 2024

Reducing inference time for a custom model

0

21

December 13, 2024

When I use TensorRT enqueueV2 it doesn't work with no error

tensorrt , cuda

0

15

December 12, 2024

How TensorRT IOPlugin get enough workspace size in explicit batch mode?

tensorrt

0

8

December 12, 2024

OCRnet Resnet 50 issue while deploying with custom character list

tensorrt , opencv , cuda , gstreamer , inference-server-triton

0

6

December 12, 2024

Can multiple CUDA contexts share an inference engine?

tensorrt , cuda

0

9

December 11, 2024

TensorRT Error: `[genericReformat.cuh::copyVectorizedRunKernel::1579] Error Code 1: Cuda Runtime (invalid resource handle)` in ROS Callback Function

tensorrt , ros , cuda , cudnn

0

15

December 10, 2024

Yolov11 Triton Inference Server Deployment Problem

tensorrt , inference-server-triton

0

24

December 7, 2024

Could not find any implementation for node

cudnn

21

163

December 6, 2024

Could not find any implementation for node failure of TensorRT 8.5 when running on GPU Jetson Xavier NX

0

11

December 6, 2024

Best websites or books on TensorRT optimization

2

23

December 5, 2024

Sparsity on Onnx Model

0

14

December 4, 2024

Int8 implementation of element-wise ops with multiple inputs

2

21

December 2, 2024

gRPC end point unavailable status on triton inference server

python , cudnn , inference-server-triton , grpc

0

7

December 2, 2024

Error adding symbols. Tensorrt installation

cudnn

2

27

December 2, 2024

Clarification on TensorRT Binding Order and Guarantee of Input-Output Sequence (bindingIsInput() etc)

tensorrt

1

28

December 2, 2024

How can I optimize multi-batch and parallel inference in TensorRT for faster performance on high-resolution image patches?

tensorrt , cuda , ubuntu , python , cudnn , deep-learning

2

56

December 2, 2024

Which version of pytorch and python is compatible with cuda 11.4

cuda , pytorch , python

0

36

November 30, 2024

Tensorrt-llm Phi-3-mini-128k-instruct error

cudnn , phi-3-mini-128k-instruct

1

44

November 30, 2024

Tensorrt Conversion

2

50

November 30, 2024

Error while deploying the Speech and NLP models for ACE

1

23

November 30, 2024

Deep Learning (Training & Inference) TensorRT