Tensorrt Code for Image Classification using nvCaffe parser.

Hi I am trying to perform Classification of Cats & Dogs using a caffe model.

To run the caffe model using tensorrt, I am using sample/MNIST.cpp as reference. I am finding difficulty in reading Image & verifying the Output. I have put the relevant pieces of Code.

Kindly help on how to get values of probability for Cats & Dogs. :)

deploy.prototxt File :


name: “CaffeNet”
layer {
name: “data”
type: “Input”
top: “data”
input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }
}
……………

layer {
name: “fc8”
type: “InnerProduct”
bottom: “fc7”
top: “fc8”
inner_product_param {
num_output: 2
}
}
layer {
name: “prob”
type: “Softmax”
bottom: “fc8”
top: “prob”
}

bool SampleMNIST::processInput(
const samplesCommon::BufferManager& buffers, const std::string& inputTensorName, int inputFileIdx) const
{
const int inputC = mInputDims.d[0]; //------------------- Added
const int inputH = mInputDims.d[1];
const int inputW = mInputDims.d[2];

const float pixelMean[3]{ 102.9801f, 115.9465f, 122.7717f }; //-----------------------Added

// Read a random digit file
srand(unsigned(time(nullptr)));
std::vector<uint8_t> fileData(inputC * inputH * inputW);
readPGMFile(locateFile(std::to_string(inputFileIdx) + ".pgm", mParams.dataDirs), fileData.data(), inputH, inputW);

// Print ASCII representation of digit
gLogInfo << "Input:\n";
for (int i = 0; i < inputC * inputH * inputW; i++)
{
    gLogInfo << (" .:-=+*#%@"[fileData[i] / 26]) << (((i + 1) % inputW) ? "" : "\n");
}
gLogInfo <<"Input H & W" << inputH << " " << inputW <<std::endl;

float* hostInputBuffer = static_cast<float*>(buffers.getHostBuffer(inputTensorName));

for (int i = 0; i < inputC *  inputH * inputW; i++)
{
    hostInputBuffer[i] = float(fileData[i]);
}


return true;

}

bool SampleMNIST::verifyOutput(
const samplesCommon::BufferManager& buffers, const std::string& outputTensorName, int groundTruthDigit) const
{
const float* prob = static_cast<const float*>(buffers.getHostBuffer(outputTensorName));
const std::vectorstd::string classes{ “cat”,“dog” };
// Print histogram of the output distribution
gLogInfo << “Output:\n”;
float val{0.0f};
int idx{0};
const int kDIGITS = 2; //Assuming 2 classes for Cats and Dogs

for (int i = 0; i < 2; i++)
{
    gLogInfo << "Probval=" << prob[i] <<" :\n";
    if (val < prob[i])
    {
        val = prob[i];
        idx = i;
    }

   //gLogInfo << i << ": " << std::string(int(std::floor(prob[i] * 10 + 0.5f)), '*') << "\n";
}
gLogInfo <<"Final Winner IDX + vAL "<<idx <<"  "<<val <<std::endl;

return (idx);

}

//I have CatsDogMeanbinaryProto. Haven’t made any changes to function
void SampleMNIST::constructNetwork(
SampleUniquePtrnvcaffeparser1::ICaffeParser& parser, SampleUniquePtrnvinfer1::INetworkDefinition& network)
{
const nvcaffeparser1::IBlobNameToTensor* blobNameToTensor = parser->parse(
mParams.prototxtFileName.c_str(), mParams.weightsFileName.c_str(), *network, nvinfer1::DataType::kFLOAT);

for (auto& s : mParams.outputTensorNames)
{
    network->markOutput(*blobNameToTensor->find(s.c_str()));
}

// add mean subtraction to the beginning of the network
nvinfer1::Dims inputDims = network->getInput(0)->getDimensions();
gLogInfo << "Input dIMS  H & W" << inputDims.d[1] << "    " << inputDims.d[2] << std::endl;
mMeanBlob
    = SampleUniquePtr<nvcaffeparser1::IBinaryProtoBlob>(parser->parseBinaryProto(mParams.meanFileName.c_str()));
nvinfer1::Weights meanWeights{nvinfer1::DataType::kFLOAT, mMeanBlob->getData(), inputDims.d[1] * inputDims.d[2]};
// For this sample, a large range based on the mean data is chosen and applied to the head of the network.
// After the mean subtraction occurs, the range is expected to be between -127 and 127, so the rest of the network
// is given a generic range.
// The preferred method is use scales computed based on a representative data set
// and apply each one individually based on the tensor. The range here is large enough for the
// network, but is chosen for example purposes only.
float maxMean
    = samplesCommon::getMaxValue(static_cast<const float*>(meanWeights.values), samplesCommon::volume(inputDims));

auto mean = network->addConstant(nvinfer1::Dims3(1, inputDims.d[1], inputDims.d[2]), meanWeights);
mean->getOutput(0)->setDynamicRange(-maxMean, maxMean);
network->getInput(0)->setDynamicRange(-maxMean, maxMean);
auto meanSub = network->addElementWise(*network->getInput(0), *mean->getOutput(0), ElementWiseOperation::kSUB);
meanSub->getOutput(0)->setDynamicRange(-maxMean, maxMean);
network->getLayer(0)->setInput(0, *meanSub->getOutput(0));
samplesCommon::setAllTensorScales(network.get(), 127.0f, 127.0f);

}

Hi,

Could you please share the script, model file and error logs so we can help better?
Also, can you provide details on the platforms you are using:
o Linux distro and version
o GPU type
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow and PyTorch version
o TensorRT version

You can also use “trtexec” command line tool for benchmarking & generating serialized engines from models.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

Thanks

Linux distro and version= None : Windowss 10
o GPU type= P2000 Quadro
o Nvidia driver version = v340.*
o CUDA version= 10.2
o CUDNN version=10.2 (I think not sure)
o Python version [if using python]= No C++ https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleMNIST/
o Tensorflow and PyTorch version= None
o TensorRT version= 7.0.0

Hi,

Please refer to below example for image classification:
https://github.com/dusty-nv/jetson-inference

Thanks

Cool… But my final OS : for running this application is gonna be QNX. Will the link provided work ?

No, https://github.com/dusty-nv/jetson-inference
this won’t help.

My final target Machine is : Nv-drive AGX with Operating System as QNX.

I have already provided the code above. My code runs but it is giving me wrong inference.

Kindly refer the taking

bool SampleMNIST::processInput(
const samplesCommon::BufferManager& buffers, const std::string& inputTensorName, int inputFileIdx) const
{
const int inputC = mInputDims.d[0]; //------------------- Added
const int inputH = mInputDims.d[1];
const int inputW = mInputDims.d[2];

const float pixelMean[3]{ 102.9801f, 115.9465f, 122.7717f }; //-----------------------Added

// Read a random digit file
srand(unsigned(time(nullptr)));
std::vector<uint8_t> fileData(inputC * inputH * inputW);
readPGMFile(locateFile(std::to_string(inputFileIdx) + “.pgm”, mParams.dataDirs), fileData.data(), inputH, inputW);

// Print ASCII representation of digit
gLogInfo << “Input:\n”;
for (int i = 0; i < inputC * inputH * inputW; i++)
{
gLogInfo << (" .:-=+*#%@"[fileData[i] / 26]) << (((i + 1) % inputW) ? “” : “\n”);
}
gLogInfo <<“Input H & W” << inputH << " " << inputW <<std::endl;

float* hostInputBuffer = static_cast<float*>(buffers.getHostBuffer(inputTensorName));

for (int i = 0; i < inputC * inputH * inputW; i++)
{
hostInputBuffer[i] = float(fileData[i]);
}

return true;
}

and Output :

bool SampleMNIST::verifyOutput(
const samplesCommon::BufferManager& buffers, const std::string& outputTensorName, int groundTruthDigit) const
{
const float* prob = static_cast<const float*>(buffers.getHostBuffer(outputTensorName));
const std::vectorstd::string classes{ “cat”,“dog” };
// Print histogram of the output distribution
gLogInfo << “Output:\n”;
float val{0.0f};
int idx{0};
const int kDIGITS = 2; //Assuming 2 classes for Cats and Dogs

for (int i = 0; i < 2; i++)
{
gLogInfo << “Probval=” << prob[i] <<" :\n";
if (val < prob[i])
{
val = prob[i];
idx = i;
}

//gLogInfo << i << ": " << std::string(int(std::floor(prob[i] * 10 + 0.5f)), ‘*’) << “\n”;
}
gLogInfo <<"Final Winner IDX + vAL “<<idx <<” "<<val <<std::endl;

return (idx);
}

In these steps or inclusion of weight’s I am doing something wrong.
Help me in figuring that out.

Thanks :)

Hi,

In that case below sample will be more relevant to your case:
https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/sampleFasterRCNN/sampleFasterRCNN.cpp

Thanks