NVIDIA Developer Forums

Can huge model with huge total weights size be infered by tensorrt at A10?

AI & Data Science Deep Learning (Training & Inference) TensorRT

spolisetty November 7, 2022, 6:24am 2

Hi,

We are unable to access the images. Could you please share with us the trtexec --verbose logs.
FYI, Please refer,

Thank you.

Topic		Replies	Views	Activity
HOW TO CONVERT a large pytorch model to TRT model? TensorRT tensorrt	7	2743	October 12, 2021
ONNX to tensorrt model file TensorRT	0	487	May 5, 2019
Run out of memory when creating TensorRT engine from onnx model Jetson Xavier NX tensorrt	7	2900	October 18, 2021
Trt file from onnx is too large TensorRT	1	937	March 10, 2021
Tensor volume exceeds (2^31)-1 TensorRT	7	2986	June 13, 2023
Tensor volume issue in onnx2tensorrt conversion process TensorRT tensorrt	1	627	April 7, 2022
Cuda OutOfMemory when creating tensor with 2^29 (~0.5 G) elements TensorRT tensorrt , cuda , onnx	6	1815	March 9, 2022
ONNX to TRT using trtexec gives output only on batch size 1 TensorRT	7	1336	November 20, 2020
AssertionError: Max workspace size for TensorRT inference should be positive, got 0 TensorRT	4	774	July 21, 2021
Myelin memory budget exceeded while building TensorRT engine with batch > 1 TensorRT tensorrt	4	1002	October 12, 2021