Multi tensorrt task issue

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hello ,
I 'm now do 2 detect task at the same time in xavier,
each takes about 200ms , and when any model runs, the gpu is all occupied.
if I want to run this two model within 300ms,
is there any solution ?

Thanks

3

Hi,

May I know which framework do you use for the inference?
If not TensorRT, it’s recommended to use it.

A common acceleration on Xavier is using INT8 precision for performance.
If your model is caffe, uff or onnx, trtexec is a good tool to test the performance for different precision .

$ /usr/src/tensorrt/bin/trtexec --onnx=[my/model] --fp16 # half mode 
$ /usr/src/tensorrt/bin/trtexec --onnx=[my/model] --int8 # int8 mode

Thanks.

5

I means that when I run 2 model at the same time,
each of them cost more time compared with running each of them alone.