TensorRT on TX1 with jetpack 2.3.1 FP16 mode support

We implement a caffe inference with tensorRT on TX1.
We checked that FP16 mode are support on current board

mEnableFP16 = (mOverride16 == true) ? false : builder->platformHasFastFp16(); 
        printf("platform %s FP16 support.\n", mEnableFP16 ? "has" : "does not have");
        printf("loading %s %s\n", deploy, caffemodel);           

        nvinfer1::DataType modelDataType = mEnableFP16 ? nvinfer1::DataType::kHALF : nvinfer1::DataType::kFLOAT;
platform has FP16 support.
loading deploy.prototxt iter_92000.caffemodel
Begin parsing probot...deploy.prototxt model: iter_92000.caffemodel
End parsing model...
Begin building engine...
const IBlobNameToTensor* blobNameToTensor = parser->parse(deployFile.c_str(),
        (modelFile).c_str(),
        *network,
        //modelDataType);
       nvinfer1::DataType::kFLOAT);
//       nvinfer1::DataType::kHALF);

I record the kHALF inference time and KFLOAT, I found that it’s almost the same. Could anybody kindly figure out what I am missing?

Hi,

Have you set inference builder to half2Mode?

builder->setHalf2Mode(true);

Thanks.

Many Thanks AastaLLL

I works for my model, it has about 40% speed up.