Hey I have been working with deep learning model for image segmentation using Tensoorrt on Jetson Xavier Nx. I am using jetpack 4.6. My question is can I deserialize and load multiple tensorrt engines to memory for inferencing ? I would also want to switch between these engine as per the input ?

Hi, YES, you can. But please make sure you have created a separated CUDA context for each model. Below is an example for your reference: [image] DLA and GPU cores at the same time Jetson AGX Xavier Hi, Here is a sample to run GPU and DLAs at the same time. 1. Please p…

TensorRt Engines On Nvidia Xavier Nx

Robotics & Edge Computing Jetson Systems Jetson Xavier NX

AastaLLL December 28, 2021, 2:53am 3

Hi,

YES, you can.
But please make sure you have created a separated CUDA context for each model.

Below is an example for your reference:

Thanks.

Topic		Replies	Views
Concurrently run two or more engine on a tensorrt TensorRT	4	2472	April 14, 2020
Deploy three AI model engines on both DLAs and GPU Jetson AGX Xavier tensorrt , jetson-inference , dla , gpu	4	727	September 26, 2023
Can multiple engines be created from the same tensorrt engine Jetson AGX Xavier tensorrt	2	408	January 9, 2024
How to parallel inference model in nvidia Xavier nx platform Jetson Xavier NX jetson-inference	7	2009	October 6, 2021
run multiple models at one time on xavier. Jetson AGX Xavier	3	2086	October 18, 2021
Run two engines in IRuntime Jetson TX2 tensorrt	2	542	October 18, 2021
Multiple tensorrt engine contexts for different models TensorRT	3	1942	March 16, 2023
Run TLT model on GPU and two DLAs simultaneously Jetson Xavier NX tensorrt	2	406	October 18, 2021
[TensorRT] Create multiple engines in one python script TensorRT	4	2366	October 11, 2018
Unable to build tensorrt engine with DLA enabled on Jetson Xavier NX Jetson Xavier NX tensorrt , cudnn	7	451	May 15, 2024

TensorRt Engines On Nvidia Xavier Nx

Related topics