What's different .trt and .engine of model?

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
1

hello,
i used tensorrt_demos to change custom yolov4.weights to onnx and int8 engine.
i got model like this: yolov4-608-int8.trt, but it can not run at test5.
if i used ./trtexec to change that i will get engine which name xxx.engine and it works well, but only fp16.
could you advice to me what is different .trt and .engine files?
how can i do ?

thanks,

2

and when i run:
./trtexec --onnx=yolov4_-1_3_608_608_dynamic.onnx --explicitBatch --saveEngine=yolov4_608.engine --best --verbose --calib=calib_yolov4-int8-608.bin

it shows error like below:

2021-09-23 06-44-52屏幕截图
2021-09-23 06-44-52屏幕截图734×462 188 KB

4

Hi,

.trt and .engine are all TensorRT engine file.
Since it is just a serialized data, the file extension varies from the name user chooses.

The error you met is from the INT8 calibrator.
Do you generate the calibration cache with YOLOv4 on Xavier directly?

Thanks.

5

hi,
i used tensorrt_demos to generate my int8.trt files with my calib-images arround 10000pcs.
i changed classes names and nums for my own model.
and the file works well when i tested it by python3, when i put this model in test5, pipline can not open model.
i try to use this new cache file of int8.bin to run above command in my trainning path: ./trtexec xxxxxxxxx --int8 --calib, that shows error.
what do you mean generate cache with yolov4 on Xavier directly?
pls give some advice to go on my works for int8…

thank you very much!!

6

Hi,

It looks like you are facing a similar issue with the following comment:

The above issue can be fixed by updating the input layer name from 000_net into data.
Could you check if the same solution also works for you?

Thanks.

7

yes it is that problem, now i get int8 works.
but new problem is why int8 inference time is slower that fp16?

int8 time
int8 time734×462 94 KB

fp16 time
fp16 time734×462 95.8 KB

8

Hi,

Have you maximized the device performance?
If not, could you do so and try it again?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

Thanks.

9

yes i used that already.
but i command is :
sudo nvpmodel -m 2
sudo jetson_clocks

10

and when i try to command:
sudo nvpmmodel -m 0
it shows like this:
NVPM WARN: patching tpc_pg_mask: (0x1:0x4)
NVPM WARN: patched tpc_pg_mask: 0x4

is this correct?

thank you

11

hi,
i tried to change trt again after fix nvpmmodel to 0,
and int8 is better than before and i can use normally right now, before i thought 2 is more power…:((((
thank you very much!

and i would like to ask a new quesiton again,
my custom model is multi objects and i have to test and modify it long time,
so sometimes i have to use yolo tiny for quickly works.
if i use 608608 on pgie and sgie, deepstream works well, but 15361536 on pgie and sgie, deepstream will be hung after 20-30 minutes, said scale problems on sgie!
could you advice a way to get it work?

thanks!

12

hi,
i tried to change trt again after fix nvpmmodel to 0,
and int8 is better than before and i can use normally right now, before i thought 2 is more power…:((((
and i would like to ask a new quesiton again,
my custom model is multi objects and i have to test and modify it long time,
so sometimes i have to use yolo tiny for quickly works.
if i use 608608 on pgie and sgie, deepstream works well, but 15361536 on pgie and sgie, deepstream will be hung after 20-30 minutes, said scale problems on sgie!
could you advice a way to get it work?

thanks!

13

Hi,

Mode 2 will enable more CPU cores but mode 0 has a higher GPU frequency.
You can find more details in the below document:

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/power_management_jetson_xavier.html#wwpID0E0PT0HA

For the new question, would you mind opening a new topic for it?
Thanks.