FP32 and FP16 imagenet

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi everyone,

I have deployed my caffe trained Alexnet on the Tensorrt imagenet and I do not experience a speed up using changing to half-precision. Both FP16 and FP32 provide me with roughly the same FPS. Can anyone help me understand why?

I thank you all,

2

Hi maycondouglasd, which TensorRT code are you using? If it is from jetson-inference, try changing this line in the code to true to disable FP16:

//mOverride16 = false;
mOverride16 = true;

You may also want to delete the previous engine’s .tensorcache if you think the changes aren’t taking effect, although this scenario should already be accommodated for.

Which program are you using to test? Try enabling TensorRT layer profiling to determine what is happening at the network level (see this line of code from imagenet-console).

3

Hi Dusty,

this was exactly the issue. By deleting the .tensorcache file the results have improved significantly.

Thanks