I’m using Jetpack on the Jetson Nano to run a TensorRT version of YOLO. I’m finding that the serialised model engine is much larger with TensorRT 7.1 (Jetpack 4.4 DP) compared to TensorRT 6.0 (Jetpack 4.3) and TensorRT 5.1 (Jetpack 4.2.1), even though the model is the same in each case.
The sizes are as follows:
TensorRT 5.1: 184 MB
TensorRT 6.0: 184 MB
TensorRT 7.1: 302 MB.
I initially thought that perhaps a kFLOAT model was being created instead of the kHALF model that I wanted. However, when I generated a kFLOAT model using TensorRT 7.1 it was 600 MB, so that doesn’t seem to be the cause of what I’m seeing. The performance of three models are similar in the few tests that I have run.
I’d appreciate it if anyone could shed light on this.