the difference of DataType::kHALF in the `parse` function of `ICaffeParser* parser` and `builder->setHalf2Mode(true)`?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

hello, I have the follow code:

ICaffeParser* parser = createCaffeParser();
    // parser->setPluginFactory(&pluginFactory);

    bool mEnableFp16 = builder->platformHasFastFp16();
    bool mEnableInt8 = builder->platformHasFastInt8();
    printf(LOG_GIE "platform %s Fp16 support.\n", mEnableFp16 ? "has" : "does not have");
    printf(LOG_GIE "platform %s Int8 support.\n", mEnableInt8 ? "has" : "does not have");

    DataType modelDataType = mEnableFp16 ? DataType::kHALF : DataType::kFLOAT;


    const IBlobNameToTensor *blobNameToTensor =	parser->parse(deployFile.c_str(),
                                                              modelFile.c_str(),
                                                              *network,
                                                              modelDataType);

    assert(blobNameToTensor != nullptr);

    for (int i = 0, n = network->getNbInputs(); i < n; i++)
    {
        Dims3 dims = static_cast<Dims3&&>(network->getInput(i)->getDimensions());
        std::cout << "Input \"" << network->getInput(i)->getName() << "\": " << dims.d[0] << "x" << dims.d[1] << "x" <<
        dims.d[2] << std::endl;
    }

    for (auto& s : outputs) network->markOutput(*blobNameToTensor->find(s.c_str()));

    builder->setMaxBatchSize(maxBatchSize);
    builder->setMaxWorkspaceSize(1 << 32);

    // set up the network for paired-fp16 format
    if(mEnableFp16)builder->setHalf2Mode(true);

I do not understand the difference of DataType::kHALF in the parse function of ICaffeParser* parser and builder->setHalf2Mode(true)?
if I use DataType::kFLOAT and not do use builder->setHalf2Mode(true), which mode will the engine infer running?

2

Hello,

DataType::kHALF will create FP16 weights. You almost certainly don’t want that, as it’s better to feed TRT 32-bit weights and let it convert once it has done all the layer fusion.

TRT will convert if you specify FP16 to the builder

3

@NVES thanks your reply. Now I see.