Could TensorRT support muilt-GPU for model converting via torch2trt

cckuo0925 September 17, 2019, 10:21am 1

Hi all,

I encounter the CUDA out of memory issue when I convert the model from pytorch to tensorRT via torch2trt package.

Is the CUDA or tensorRT provide the such arguments which let the converting operation could use the muilt-GPU RAM.

My GPU device is RTX2080ti
CUDA version :10.0
CuDNN version : 7.5
TensorRT version :5.1.5.0

Thanks!

Topic		Replies	Views
How to inference with tensorrt on multi gpus in python TensorRT	2	2302	April 9, 2021
New TensorRT Model occupying more GPU Memory as compared to older version TensorRT tensorrt , tensorflow , gpu	8	2166	August 20, 2021
Multi model inference - Swap GPU memory TensorRT	3	664	February 19, 2021
PyTorch model converting to TensorRT issue Jetson TX2	3	1211	October 2, 2019
Multiple TensorRT models Inference on Jetson Orin Jetson AGX Orin tensorrt	1	409	April 24, 2024
Multi-GPU inference from caffemodel in TensorRT GPU-Accelerated Libraries	0	893	January 31, 2018
How to convert TensortFlow to TensorRT with two ouputs TensorRT	2	728	September 16, 2020
Using Multiple GPUS TensorRT	0	600	August 20, 2019
How much GPU memory does TensorRT need to convert a model (e.g. Llama 7b with fp16) TensorRT	2	1539	November 4, 2023
How convert pytorch model that have mutiple parallel inputs to tensorrt ？ TensorRT	1	810	June 14, 2022