Feasibility of SSD, YOLO models on TensorRT and Deepstream?

Description

I can convert a general ResNet50.caffemodel to ResNet50.engile file to be used in TensorRT and Deepstream. I want to know how to do the same for SSD model,Tiny YoloV3, YoloV2,YoloV3, SSD_mobileNet_v2,REID model.
I have tried with the RESNET50 conversion sample but did not get the answer. Script I used was in /usr/src/tensorrt/samples/python/introductory_parser_samples/caffe_resnet50.py

Kindly give support for the same.

Environment

**TensorRT **: Version5.1.5
GPU Type: NVIDIA T4
Nvidia Driver Version:
CUDA Version release 10.1, V10.1.105:
nvidia-smi: NVIDIA-SMI 450.36.06 Driver Version: 450.36.06
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): Python 2.7.17 (default, Apr 15 2020, 17:20:14)
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Hi @GokuDDG,

Can you re-try after upgrading to latest TRT release, try trtexec cmd line tool.
Please share the verbose error along with model file if the issue persist.

Please find the sample examples for the listed model to trt conversion.
https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html

Thanks!

https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html
I had gone through the steps and in /usr/src/tensorrt/samples/sampleSSD/sampleSSD.cpp
1)we are just taking creating an engine and giving inference. I actually wanted to store the engine(serialize)
Where should I implement in the sampleSSD.cpp I had gotten a snippet for the same.
IHostMemory serializedModel = engine->serialize();
// store model to disk
// <…>
serializedModel->destroy();
IRuntime
runtime = createInferRuntime(gLogger);
ICudaEngine* engine = runtime->deserializeCudaEngine(modelData, modelSize, nullptr);
Kindly help me to implement this ssd model and to save it to .Engine format.
Hope this is the best way to convert the model to be used on Deepstream.

Hi @GokuDDG,
Can you please try following the links in sequence.
Upgrade the trt version to the latest available release to avoid any minor issues.
Follow the link on building the engine


Next is to serialize your model and saving it to the disk.
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#serial_model_c

Hope this helps.
Thanks!

Hi @AakankshaS ,
I had gone through that link. Can you help me how and where we should implement this
IHostMemory serializedModel = engine->serialize();
// store model to disk
// <…>
serializedModel->destroy();
IRuntime
runtime = createInferRuntime(gLogger);
ICudaEngine* engine = runtime->deserializeCudaEngine(modelData, modelSize, nullptr);
in sampleSSD.cpp

Hi @GokuDDG,
Apologies, i just missed to paste the reference link.
Please refer to this code.


Thanks!

Hi @AakankshaS,
I have attached image of error I recieved after editing with your snippet. Kindly give your expertise and assistance on the matter.


Thanks in Advance
Here giving the edited samplemnist.cpp file content
/*

  • Copyright 1993-2019 NVIDIA Corporation. All rights reserved.
  • NOTICE TO LICENSEE:
  • This source code and/or documentation (“Licensed Deliverables”) are
  • subject to NVIDIA intellectual property rights under U.S. and
  • international Copyright laws.
  • These Licensed Deliverables contained herein is PROPRIETARY and
  • CONFIDENTIAL to NVIDIA and is being provided under the terms and
  • conditions of a form of NVIDIA software license agreement by and
  • between NVIDIA and Licensee (“License Agreement”) or electronically
  • accepted by Licensee. Notwithstanding any terms or conditions to
  • the contrary in the License Agreement, reproduction or disclosure
  • of the Licensed Deliverables to any third party without the express
  • written consent of NVIDIA is prohibited.
  • NOTWITHSTANDING ANY TERMS OR CONDITIONS TO THE CONTRARY IN THE
  • LICENSE AGREEMENT, NVIDIA MAKES NO REPRESENTATION ABOUT THE
  • SUITABILITY OF THESE LICENSED DELIVERABLES FOR ANY PURPOSE. IT IS
  • PROVIDED “AS IS” WITHOUT EXPRESS OR IMPLIED WARRANTY OF ANY KIND.
  • NVIDIA DISCLAIMS ALL WARRANTIES WITH REGARD TO THESE LICENSED
  • DELIVERABLES, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY,
  • NONINFRINGEMENT, AND FITNESS FOR A PARTICULAR PURPOSE.
  • NOTWITHSTANDING ANY TERMS OR CONDITIONS TO THE CONTRARY IN THE
  • LICENSE AGREEMENT, IN NO EVENT SHALL NVIDIA BE LIABLE FOR ANY
  • SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, OR ANY
  • DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
  • WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS
  • ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
  • OF THESE LICENSED DELIVERABLES.
  • U.S. Government End Users. These Licensed Deliverables are a
  • “commercial item” as that term is defined at 48 C.F.R. 2.101 (OCT
  • 1995), consisting of “commercial computer software” and "commercial
  • computer software documentation" as such terms are used in 48
  • C.F.R. 12.212 (SEPT 1995) and is provided to the U.S. Government
  • only as a commercial end item. Consistent with 48 C.F.R.12.212 and
  • 48 C.F.R. 227.7202-1 through 227.7202-4 (JUNE 1995), all
  • U.S. Government End Users acquire the Licensed Deliverables with
  • only those rights set forth herein.
  • Any use of the Licensed Deliverables in individual and commercial
  • software must include, in the user documentation and internal
  • comments to the code, the above Disclaimer and U.S. Government End
  • Users Notice.
    */

//! \file sampleMNIST.cpp
//! \brief This file contains the implementation of the MNIST sample.
//!
//! It builds a TensorRT engine by importing a trained MNIST Caffe model. It uses the engine to run
//! inference on an input image of a digit.
//! It can be run with the following command line:
//! Command: ./sample_mnist [-h or --help] [-d=/path/to/data/dir or --datadir=/path/to/data/dir]

#include “logger.h”
#include “common.h”
#include “argsParser.h”
#include “buffers.h”

#include “NvCaffeParser.h”
#include “NvInfer.h”

#include <cuda_runtime_api.h>
#include
#include
#include
#include
#include
#include

const std::string gSampleName = “TensorRT.sample_mnist”;

//!
//! \brief The MNISTSampleParams structure groups the additional parameters required by
//! the MNIST sample.
//!
struct MNISTSampleParams : public samplesCommon::CaffeSampleParams
{
std::string meanFileName;
};

//!
//! \brief The SampleMNIST class implements the MNIST sample
//!
//! \details It creates the network using a trained Caffe MNIST classification model
//!
class SampleMNIST
{
template
using SampleUniquePtr = std::unique_ptr<T, samplesCommon::InferDeleter>;

public:
SampleMNIST(const MNISTSampleParams& params)
: mParams(params)
{
}

//!
//! \brief Function builds the network engine
//!
bool build();

//!
//! \brief This function runs the TensorRT inference engine for this sample
//!
bool infer();

//!
//! \brief This function can be used to clean up any state created in the sample class
//!
bool teardown();

private:
//!
//! \brief This function uses a Caffe parser to create the MNIST Network and marks the
//! output layers
//!
void constructNetwork(SampleUniquePtrnvinfer1::IBuilder& builder, SampleUniquePtrnvinfer1::INetworkDefinition& network, SampleUniquePtrnvcaffeparser1::ICaffeParser& parser);

//!
//! \brief Reads the input and mean data, preprocesses, and stores the result in a managed buffer
//!
bool processInput(const samplesCommon::BufferManager& buffers, const std::string& inputTensorName, int inputFileIdx) const;

//!
//! \brief Verifies that the output is correct and prints it
//!
bool verifyOutput(const samplesCommon::BufferManager& buffers, const std::string& outputTensorName, int groundTruthDigit) const;

std::shared_ptr<nvinfer1::ICudaEngine> mEngine = nullptr; //!< The TensorRT engine used to run the network

MNISTSampleParams mParams; //!< The parameters for the sample.

nvinfer1::Dims mInputDims; //!< The dimensions of the input to the network.

SampleUniquePtr<nvcaffeparser1::IBinaryProtoBlob> mMeanBlob; //! the mean blob, which we need to keep around until build is done

};

//!
//! \brief This function creates the network, configures the builder and creates the network engine
//!
//! \details This function creates the MNIST network by parsing the caffe model and builds
//! the engine that will be used to run MNIST (mEngine)
//!
//! \return Returns true if the engine was created successfully and false otherwise
//!
bool SampleMNIST::build()
{
auto builder = SampleUniquePtrnvinfer1::IBuilder(nvinfer1::createInferBuilder(gLogger.getTRTLogger()));
if (!builder)
return false;

auto network = SampleUniquePtr<nvinfer1::INetworkDefinition>(builder->createNetwork());
if (!network)
    return false;

auto parser = SampleUniquePtr<nvcaffeparser1::ICaffeParser>(nvcaffeparser1::createCaffeParser());
if (!parser)
    return false;

constructNetwork(builder, network, parser);
builder->setMaxBatchSize(mParams.batchSize);
builder->setMaxWorkspaceSize(16_MB);
builder->allowGPUFallback(true);
builder->setFp16Mode(mParams.fp16);
builder->setInt8Mode(mParams.int8);
builder->setStrictTypeConstraints(true);
builder->setInt8Calibrator(calibrator.get());

samplesCommon::enableDLA(builder.get(), mParams.dlaCore);

mEngine = std::shared_ptr<nvinfer1::ICudaEngine>(builder->buildCudaEngine(*network), samplesCommon::InferDeleter());

if (!mEngine)
    return false;

if (mParams.saveEngine.size() > 0)
{
   std::ofstream p(mParams.saveEngine, std::ios::binary);
   if (!p)
   {
       return false;
   }
   nvinfer1::IHostMemory* ptr = mEngine->serialize();
   assert(ptr);
   p.write(reinterpret_cast<const char*>(ptr->data()), ptr->size());
   ptr->destroy();
}
assert(network->getNbInputs() == 1);
mInputDims = network->getInput(0)->getDimensions();
assert(mInputDims.nbDims == 3);

return true;

}

//!
//! \brief Reads the input and mean data, preprocesses, and stores the result in a managed buffer
//!
bool SampleMNIST::processInput(const samplesCommon::BufferManager& buffers, const std::string& inputTensorName, int inputFileIdx) const
{
const int inputH = mInputDims.d[1];
const int inputW = mInputDims.d[2];

// Read a random digit file
srand(unsigned(time(nullptr)));
std::vector<uint8_t> fileData(inputH * inputW);
readPGMFile(locateFile(std::to_string(inputFileIdx) + ".pgm", mParams.dataDirs), fileData.data(), inputH, inputW);

// Print ASCII representation of digit
gLogInfo << "Input:\n";
for (int i = 0; i < inputH * inputW; i++)
    gLogInfo << (" .:-=+*#%@"[fileData[i] / 26]) << (((i + 1) % inputW) ? "" : "\n");
gLogInfo << std::endl;

float* hostInputBuffer = static_cast<float*>(buffers.getHostBuffer(inputTensorName));

name: “VGG_VOC0712_SSD_300x300_deploy”
input: “data”
input_shape {
dim: 1
dim: 3
dim: 300
dim: 300
}
layer {
name: “conv1_1”
type: “Convolution”
bottom: “data”
top: “conv1_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu1_1”
type: “ReLU”
bottom: “conv1_1”
top: “conv1_1”
}
layer {
name: “conv1_2”
type: “Convolution”
bottom: “conv1_1”
top: “conv1_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu1_2”
type: “ReLU”
bottom: “conv1_2”
top: “conv1_2”
}
layer {
name: “pool1”
type: “Pooling”
bottom: “conv1_2”
top: “pool1”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv2_1”
type: “Convolution”
bottom: “pool1”
top: “conv2_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu2_1”
type: “ReLU”
bottom: “conv2_1”
top: “conv2_1”
}
layer {
name: “conv2_2”
type: “Convolution”
bottom: “conv2_1”
top: “conv2_2”
param {
lr_mult: 1
decay_mult: 1name: “VGG_VOC0712_SSD_300x300_deploy”
input: “data”
input_shape {
dim: 1
dim: 3
dim: 300
dim: 300
}
layer {
name: “conv1_1”
type: “Convolution”
bottom: “data”
top: “conv1_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu1_1”
type: “ReLU”
bottom: “conv1_1”
top: “conv1_1”
}
layer {
name: “conv1_2”
type: “Convolution”
bottom: “conv1_1”
top: “conv1_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu1_2”
type: “ReLU”
bottom: “conv1_2”
top: “conv1_2”
}
layer {
name: “pool1”
type: “Pooling”
bottom: “conv1_2”
top: “pool1”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv2_1”
type: “Convolution”
bottom: “pool1”
top: “conv2_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu2_1”
type: “ReLU”
bottom: “conv2_1”
top: “conv2_1”
}
layer {
name: “conv2_2”
type: “Convolution”
bottom: “conv2_1”
top: “conv2_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu2_2”
type: “ReLU”
bottom: “conv2_2”
top: “conv2_2”
}
layer {
name: “pool2”
type: “Pooling”
bottom: “conv2_2”
top: “pool2”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv3_1”
type: “Convolution”
bottom: “pool2”
top: “conv3_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_1”
type: “ReLU”
bottom: “conv3_1”
top: “conv3_1”
}
layer {
name: “conv3_2”
type: “Convolution”
bottom: “conv3_1”
top: “conv3_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_2”
type: “ReLU”
bottom: “conv3_2”
top: “conv3_2”
}
layer {
name: “conv3_3”
type: “Convolution”
bottom: “conv3_2”
top: “conv3_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_3”
type: “ReLU”
bottom: “conv3_3”
top: “conv3_3”
}
layer {
name: “pool3”
type: “Pooling”
bottom: “conv3_3”
top: “pool3”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv4_1”
type: “Convolution”
bottom: “pool3”
top: “conv4_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_1”
type: “ReLU”
bottom: “conv4_1”
top: “conv4_1”
}
layer {
name: “conv4_2”
type: “Convolution”
bottom: “conv4_1”
top: “conv4_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_2”
type: “ReLU”
bottom: “conv4_2”
top: “conv4_2”
}
layer {
name: “conv4_3”
type: “Convolution”
bottom: “conv4_2”
top: “conv4_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_3”
type: “ReLU”
bottom: “conv4_3”
top: “conv4_3”
}
layer {
name: “pool4”
type: “Pooling”
bottom: “conv4_3”
top: “pool4”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv5_1”
type: “Convolution”
bottom: “pool4”
top: “conv5_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_1”
type: “ReLU”
bottom: “conv5_1”
top: “conv5_1”
}
layer {
name: “conv5_2”
type: “Convolution”
bottom: “conv5_1”
top: “conv5_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_2”
type: “ReLU”
bottom: “conv5_2”
top: “conv5_2”
}
layer {
name: “conv5_3”
type: “Convolution”
bottom: “conv5_2”
top: “conv5_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_3”
type: “ReLU”
bottom: “conv5_3”
top: “conv5_3”
}
layer {
name: “pool5”
type: “Pooling”
bottom: “conv5_3”
top: “pool5”
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}
layer {
name: “fc6”
type: “Convolution”
bottom: “pool5”
top: “fc6”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1024
pad: 6
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 6
}
}
layer {
name: “relu6”
type: “ReLU”
bottom: “fc6”
top: “fc6”
}
layer {
name: “fc7”
type: “Convolution”
bottom: “fc6”
top: “fc7”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1024
kernel_size: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu7”
type: “ReLU”
bottom: “fc7”
top: “fc7”
}
layer {
name: “conv6_1”
type: “Convolution”
bottom: “fc7”
top: “conv6_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_1_relu”
type: “ReLU”
bottom: “conv6_1”
top: “conv6_1”
}
layer {
name: “conv6_2”
type: “Convolution”
bottom: “conv6_1”
top: “conv6_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 2
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_relu”
type: “ReLU”
bottom: “conv6_2”
top: “conv6_2”
}
layer {
name: “conv7_1”
type: “Convolution”
bottom: “conv6_2”
top: “conv7_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_1_relu”
type: “ReLU”
bottom: “conv7_1”
top: “conv7_1”
}
layer {
name: “conv7_2”
type: “Convolution”
bottom: “conv7_1”
top: “conv7_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 2
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_relu”
type: “ReLU”
bottom: “conv7_2”
top: “conv7_2”
}
layer {
name: “conv8_1”
type: “Convolution”
bottom: “conv7_2”
top: “conv8_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_1_relu”
type: “ReLU”
bottom: “conv8_1”
top: “conv8_1”
}
layer {
name: “conv8_2”
type: “Convolution”
bottom: “conv8_1”
top: “conv8_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_relu”
type: “ReLU”
bottom: “conv8_2”
top: “conv8_2”
}
layer {
name: “conv9_1”
type: “Convolution”
bottom: “conv8_2”
top: “conv9_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_1_relu”
type: “ReLU”
bottom: “conv9_1”
top: “conv9_1”
}
layer {
name: “conv9_2”
type: “Convolution”
bottom: “conv9_1”
top: “conv9_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_relu”
type: “ReLU”
bottom: “conv9_2”
top: “conv9_2”
}
layer {
name: “conv4_3_norm”
type: “Normalize”
bottom: “conv4_3”
top: “conv4_3_norm”
norm_param {
across_spatial: false
scale_filler {
type: “constant”
value: 20
}
channel_shared: false
}
}
layer {
name: “conv4_3_norm_mbox_loc”
type: “Convolution”
bottom: “conv4_3_norm”
top: “conv4_3_norm_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv4_3_norm_mbox_loc_perm”
type: “Permute”
bottom: “conv4_3_norm_mbox_loc”
top: “conv4_3_norm_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv4_3_norm_mbox_loc_flat”
type: “Reshape”
bottom: “conv4_3_norm_mbox_loc_perm”
top: “conv4_3_norm_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv4_3_norm_mbox_conf”
type: “Convolution”
bottom: “conv4_3_norm”
top: “conv4_3_norm_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv4_3_norm_mbox_conf_perm”
type: “Permute”
bottom: “conv4_3_norm_mbox_conf”
top: “conv4_3_norm_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv4_3_norm_mbox_conf_flat”
type: “Reshape”
bottom: “conv4_3_norm_mbox_conf_perm”
top: “conv4_3_norm_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv4_3_norm_mbox_priorbox”
type: “PriorBox”
bottom: “conv4_3_norm”
bottom: “data”
top: “conv4_3_norm_mbox_priorbox”
prior_box_param {
min_size: 30.0
max_size: 60.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 8
offset: 0.5
}
}
layer {
name: “fc7_mbox_loc”
type: “Convolution”
bottom: “fc7”
top: “fc7_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “fc7_mbox_loc_perm”
type: “Permute”
bottom: “fc7_mbox_loc”
top: “fc7_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “fc7_mbox_loc_flat”
type: “Reshape”
bottom: “fc7_mbox_loc_perm”
top: “fc7_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “fc7_mbox_conf”
type: “Convolution”
bottom: “fc7”
top: “fc7_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “fc7_mbox_conf_perm”
type: “Permute”
bottom: “fc7_mbox_conf”
top: “fc7_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “fc7_mbox_conf_flat”
type: “Reshape”
bottom: “fc7_mbox_conf_perm”
top: “fc7_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “fc7_mbox_priorbox”
type: “PriorBox”
bottom: “fc7”
bottom: “data”
top: “fc7_mbox_priorbox”
prior_box_param {
min_size: 60.0
max_size: 111.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 16
offset: 0.5
}
}
layer {
name: “conv6_2_mbox_loc”
type: “Convolution”
bottom: “conv6_2”
top: “conv6_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_mbox_loc_perm”
type: “Permute”
bottom: “conv6_2_mbox_loc”
top: “conv6_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv6_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv6_2_mbox_loc_perm”
top: “conv6_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv6_2_mbox_conf”
type: “Convolution”
bottom: “conv6_2”
top: “conv6_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_mbox_conf_perm”
type: “Permute”
bottom: “conv6_2_mbox_conf”
top: “conv6_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv6_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv6_2_mbox_conf_perm”
top: “conv6_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv6_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv6_2”
bottom: “data”
top: “conv6_2_mbox_priorbox”
prior_box_param {
min_size: 111.0
max_size: 162.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 32
offset: 0.5
}
}
layer {
name: “conv7_2_mbox_loc”
type: “Convolution”
bottom: “conv7_2”
top: “conv7_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_mbox_loc_perm”
type: “Permute”
bottom: “conv7_2_mbox_loc”
top: “conv7_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv7_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv7_2_mbox_loc_perm”
top: “conv7_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv7_2_mbox_conf”
type: “Convolution”
bottom: “conv7_2”
top: “conv7_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_mbox_conf_perm”
type: “Permute”
bottom: “conv7_2_mbox_conf”
top: “conv7_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv7_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv7_2_mbox_conf_perm”
top: “conv7_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv7_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv7_2”
bottom: “data”
top: “conv7_2_mbox_priorbox”
prior_box_param {
min_size: 162.0
max_size: 213.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 64
offset: 0.5
}
}
layer {
name: “conv8_2_mbox_loc”
type: “Convolution”
bottom: “conv8_2”
top: “conv8_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_mbox_loc_perm”
type: “Permute”
bottom: “conv8_2_mbox_loc”
top: “conv8_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv8_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv8_2_mbox_loc_perm”
top: “conv8_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv8_2_mbox_conf”
type: “Convolution”
bottom: “conv8_2”
top: “conv8_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_mbox_conf_perm”
type: “Permute”
bottom: “conv8_2_mbox_conf”
top: “conv8_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv8_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv8_2_mbox_conf_perm”
top: “conv8_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv8_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv8_2”
bottom: “data”
top: “conv8_2_mbox_priorbox”
prior_box_param {
min_size: 213.0
max_size: 264.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 100
offset: 0.5
}
}
layer {
name: “conv9_2_mbox_loc”
type: “Convolution”
bottom: “conv9_2”
top: “conv9_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_mbox_loc_perm”
type: “Permute”
bottom: “conv9_2_mbox_loc”
top: “conv9_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv9_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv9_2_mbox_loc_perm”
top: “conv9_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv9_2_mbox_conf”
type: “Convolution”
bottom: “conv9_2”
top: “conv9_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_mbox_conf_perm”
type: “Permute”
bottom: “conv9_2_mbox_conf”
top: “conv9_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv9_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv9_2_mbox_conf_perm”
top: “conv9_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv9_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv9_2”
bottom: “data”
top: “conv9_2_mbox_priorbox”
prior_box_param {
min_size: 264.0
max_size: 315.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 300
offset: 0.5
}
}
layer {
name: “mbox_loc”
type: “Concat”
bottom: “conv4_3_norm_mbox_loc_flat”
bottom: “fc7_mbox_loc_flat”
bottom: “conv6_2_mbox_loc_flat”
bottom: “conv7_2_mbox_loc_flat”
bottom: “conv8_2_mbox_loc_flat”
bottom: “conv9_2_mbox_loc_flat”
top: “mbox_loc”
concat_param {
axis: 1
}
}
layer {
name: “mbox_conf”
type: “Concat”
bottom: “conv4_3_norm_mbox_conf_flat”
bottom: “fc7_mbox_conf_flat”
bottom: “conv6_2_mbox_conf_flat”
bottom: “conv7_2_mbox_conf_flat”
bottom: “conv8_2_mbox_conf_flat”
bottom: “conv9_2_mbox_conf_flat”
top: “mbox_conf”
concat_param {
axis: 1
}
}
layer {
name: “mbox_priorbox”
type: “Concat”
bottom: “conv4_3_norm_mbox_priorbox”
bottom: “fc7_mbox_priorbox”
bottom: “conv6_2_mbox_priorbox”
bottom: “conv7_2_mbox_priorbox”
bottom: “conv8_2_mbox_priorbox”
bottom: “conv9_2_mbox_priorbox”
top: “mbox_priorbox”
concat_param {
axis: 2
}
}
layer {
name: “mbox_conf_reshape”
type: “Reshape”
bottom: “mbox_conf”
top: “mbox_conf_reshape”
reshape_param {
shape {
dim: 0
dim: -1
dim: 21
}
}
}
layer {
name: “mbox_conf_softmax”
type: “Softmax”
bottom: “mbox_conf_reshape”
top: “mbox_conf_softmax”
softmax_param {
axis: 2
}
}
layer {
name: “mbox_conf_flatten”
type: “Reshape”
bottom: “mbox_conf_softmax”
top: “mbox_conf_flatten”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “detection_out”
type: “DetectionOutput”
bottom: “mbox_loc”
bottom: “mbox_conf_flatten”
bottom: “mbox_priorbox”
top: “detection_out”
include {
phase: TEST
}
detection_output_param {
num_classes: 21
share_location: true
background_label_id: 0
nms_param {
nms_threshold: 0.45
top_k: 400
}
save_output_param {
label_map_file: “data/VOC0712/labelmap_voc.prototxt”
}
code_type: CENTER_SIZE
keep_top_k: 200
confidence_threshold: 0.01
}
}

}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu2_2”
type: “ReLU”
bottom: “conv2_2”
top: “conv2_2”
}
layer {
name: “pool2”
type: “Pooling”
bottom: “conv2_2”
top: “pool2”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv3_1”
type: “Convolution”
bottom: “pool2”
top: “conv3_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_1”
type: “ReLU”
bottom: “conv3_1”
top: “conv3_1”
}
layer {
name: “conv3_2”
type: “Convolution”
bottom: “conv3_1”
top: “conv3_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_2”
type: “ReLU”
bottom: “conv3_2”
top: “conv3_2”
}
layer {
name: “conv3_3”
type: “Convolution”
bottom: “conv3_2”
top: “conv3_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu3_3”
type: “ReLU”
bottom: “conv3_3”
top: “conv3_3”
}
layer {
name: “pool3”
type: “Pooling”
bottom: “conv3_3”
top: “pool3”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv4_1”
type: “Convolution”
bottom: “pool3”
top: “conv4_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_1”
type: “ReLU”
bottom: “conv4_1”
top: “conv4_1”
}
layer {
name: “conv4_2”
type: “Convolution”
bottom: “conv4_1”
top: “conv4_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_2”
type: “ReLU”
bottom: “conv4_2”
top: “conv4_2”
}
layer {
name: “conv4_3”
type: “Convolution”
bottom: “conv4_2”
top: “conv4_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu4_3”
type: “ReLU”
bottom: “conv4_3”
top: “conv4_3”
}
layer {
name: “pool4”
type: “Pooling”
bottom: “conv4_3”
top: “pool4”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv5_1”
type: “Convolution”
bottom: “pool4”
top: “conv5_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_1”
type: “ReLU”
bottom: “conv5_1”
top: “conv5_1”
}
layer {
name: “conv5_2”
type: “Convolution”
bottom: “conv5_1”
top: “conv5_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_2”
type: “ReLU”
bottom: “conv5_2”
top: “conv5_2”
}
layer {
name: “conv5_3”
type: “Convolution”
bottom: “conv5_2”
top: “conv5_3”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 1
}
}
layer {
name: “relu5_3”
type: “ReLU”
bottom: “conv5_3”
top: “conv5_3”
}
layer {
name: “pool5”
type: “Pooling”
bottom: “conv5_3”
top: “pool5”
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}
layer {
name: “fc6”
type: “Convolution”
bottom: “pool5”
top: “fc6”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1024
pad: 6
kernel_size: 3
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
dilation: 6
}
}
layer {
name: “relu6”
type: “ReLU”
bottom: “fc6”
top: “fc6”
}
layer {
name: “fc7”
type: “Convolution”
bottom: “fc6”
top: “fc7”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1024
kernel_size: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “relu7”
type: “ReLU”
bottom: “fc7”
top: “fc7”
}
layer {
name: “conv6_1”
type: “Convolution”
bottom: “fc7”
top: “conv6_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_1_relu”
type: “ReLU”
bottom: “conv6_1”
top: “conv6_1”
}
layer {
name: “conv6_2”
type: “Convolution”
bottom: “conv6_1”
top: “conv6_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 2
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_relu”
type: “ReLU”
bottom: “conv6_2”
top: “conv6_2”
}
layer {
name: “conv7_1”
type: “Convolution”
bottom: “conv6_2”
top: “conv7_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_1_relu”
type: “ReLU”
bottom: “conv7_1”
top: “conv7_1”
}
layer {
name: “conv7_2”
type: “Convolution”
bottom: “conv7_1”
top: “conv7_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 2
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_relu”
type: “ReLU”
bottom: “conv7_2”
top: “conv7_2”
}
layer {
name: “conv8_1”
type: “Convolution”
bottom: “conv7_2”
top: “conv8_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_1_relu”
type: “ReLU”
bottom: “conv8_1”
top: “conv8_1”
}
layer {
name: “conv8_2”
type: “Convolution”
bottom: “conv8_1”
top: “conv8_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_relu”
type: “ReLU”
bottom: “conv8_2”
top: “conv8_2”
}
layer {
name: “conv9_1”
type: “Convolution”
bottom: “conv8_2”
top: “conv9_1”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_1_relu”
type: “ReLU”
bottom: “conv9_1”
top: “conv9_1”
}
layer {
name: “conv9_2”
type: “Convolution”
bottom: “conv9_1”
top: “conv9_2”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 0
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_relu”
type: “ReLU”
bottom: “conv9_2”
top: “conv9_2”
}
layer {
name: “conv4_3_norm”
type: “Normalize”
bottom: “conv4_3”
top: “conv4_3_norm”
norm_param {
across_spatial: false
scale_filler {
type: “constant”
value: 20
}
channel_shared: false
}
}
layer {
name: “conv4_3_norm_mbox_loc”
type: “Convolution”
bottom: “conv4_3_norm”
top: “conv4_3_norm_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv4_3_norm_mbox_loc_perm”
type: “Permute”
bottom: “conv4_3_norm_mbox_loc”
top: “conv4_3_norm_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv4_3_norm_mbox_loc_flat”
type: “Reshape”
bottom: “conv4_3_norm_mbox_loc_perm”
top: “conv4_3_norm_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv4_3_norm_mbox_conf”
type: “Convolution”
bottom: “conv4_3_norm”
top: “conv4_3_norm_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv4_3_norm_mbox_conf_perm”
type: “Permute”
bottom: “conv4_3_norm_mbox_conf”
top: “conv4_3_norm_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv4_3_norm_mbox_conf_flat”
type: “Reshape”
bottom: “conv4_3_norm_mbox_conf_perm”
top: “conv4_3_norm_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv4_3_norm_mbox_priorbox”
type: “PriorBox”
bottom: “conv4_3_norm”
bottom: “data”
top: “conv4_3_norm_mbox_priorbox”
prior_box_param {
min_size: 30.0
max_size: 60.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 8
offset: 0.5
}
}
layer {
name: “fc7_mbox_loc”
type: “Convolution”
bottom: “fc7”
top: “fc7_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “fc7_mbox_loc_perm”
type: “Permute”
bottom: “fc7_mbox_loc”
top: “fc7_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “fc7_mbox_loc_flat”
type: “Reshape”
bottom: “fc7_mbox_loc_perm”
top: “fc7_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “fc7_mbox_conf”
type: “Convolution”
bottom: “fc7”
top: “fc7_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “fc7_mbox_conf_perm”
type: “Permute”
bottom: “fc7_mbox_conf”
top: “fc7_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “fc7_mbox_conf_flat”
type: “Reshape”
bottom: “fc7_mbox_conf_perm”
top: “fc7_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “fc7_mbox_priorbox”
type: “PriorBox”
bottom: “fc7”
bottom: “data”
top: “fc7_mbox_priorbox”
prior_box_param {
min_size: 60.0
max_size: 111.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 16
offset: 0.5
}
}
layer {
name: “conv6_2_mbox_loc”
type: “Convolution”
bottom: “conv6_2”
top: “conv6_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_mbox_loc_perm”
type: “Permute”
bottom: “conv6_2_mbox_loc”
top: “conv6_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv6_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv6_2_mbox_loc_perm”
top: “conv6_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv6_2_mbox_conf”
type: “Convolution”
bottom: “conv6_2”
top: “conv6_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv6_2_mbox_conf_perm”
type: “Permute”
bottom: “conv6_2_mbox_conf”
top: “conv6_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv6_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv6_2_mbox_conf_perm”
top: “conv6_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv6_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv6_2”
bottom: “data”
top: “conv6_2_mbox_priorbox”
prior_box_param {
min_size: 111.0
max_size: 162.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 32
offset: 0.5
}
}
layer {
name: “conv7_2_mbox_loc”
type: “Convolution”
bottom: “conv7_2”
top: “conv7_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 24
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_mbox_loc_perm”
type: “Permute”
bottom: “conv7_2_mbox_loc”
top: “conv7_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv7_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv7_2_mbox_loc_perm”
top: “conv7_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv7_2_mbox_conf”
type: “Convolution”
bottom: “conv7_2”
top: “conv7_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 126
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv7_2_mbox_conf_perm”
type: “Permute”
bottom: “conv7_2_mbox_conf”
top: “conv7_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv7_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv7_2_mbox_conf_perm”
top: “conv7_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv7_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv7_2”
bottom: “data”
top: “conv7_2_mbox_priorbox”
prior_box_param {
min_size: 162.0
max_size: 213.0
aspect_ratio: 2
aspect_ratio: 3
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 64
offset: 0.5
}
}
layer {
name: “conv8_2_mbox_loc”
type: “Convolution”
bottom: “conv8_2”
top: “conv8_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_mbox_loc_perm”
type: “Permute”
bottom: “conv8_2_mbox_loc”
top: “conv8_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv8_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv8_2_mbox_loc_perm”
top: “conv8_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv8_2_mbox_conf”
type: “Convolution”
bottom: “conv8_2”
top: “conv8_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv8_2_mbox_conf_perm”
type: “Permute”
bottom: “conv8_2_mbox_conf”
top: “conv8_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv8_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv8_2_mbox_conf_perm”
top: “conv8_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv8_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv8_2”
bottom: “data”
top: “conv8_2_mbox_priorbox”
prior_box_param {
min_size: 213.0
max_size: 264.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 100
offset: 0.5
}
}
layer {
name: “conv9_2_mbox_loc”
type: “Convolution”
bottom: “conv9_2”
top: “conv9_2_mbox_loc”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_mbox_loc_perm”
type: “Permute”
bottom: “conv9_2_mbox_loc”
top: “conv9_2_mbox_loc_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv9_2_mbox_loc_flat”
type: “Reshape”
bottom: “conv9_2_mbox_loc_perm”
top: “conv9_2_mbox_loc_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv9_2_mbox_conf”
type: “Convolution”
bottom: “conv9_2”
top: “conv9_2_mbox_conf”
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 84
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
value: 0
}
}
}
layer {
name: “conv9_2_mbox_conf_perm”
type: “Permute”
bottom: “conv9_2_mbox_conf”
top: “conv9_2_mbox_conf_perm”
permute_param {
order: 0
order: 2
order: 3
order: 1
}
}
layer {
name: “conv9_2_mbox_conf_flat”
type: “Reshape”
bottom: “conv9_2_mbox_conf_perm”
top: “conv9_2_mbox_conf_flat”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “conv9_2_mbox_priorbox”
type: “PriorBox”
bottom: “conv9_2”
bottom: “data”
top: “conv9_2_mbox_priorbox”
prior_box_param {
min_size: 264.0
max_size: 315.0
aspect_ratio: 2
flip: true
clip: false
variance: 0.1
variance: 0.1
variance: 0.2
variance: 0.2
step: 300
offset: 0.5
}
}
layer {
name: “mbox_loc”
type: “Concat”
bottom: “conv4_3_norm_mbox_loc_flat”
bottom: “fc7_mbox_loc_flat”
bottom: “conv6_2_mbox_loc_flat”
bottom: “conv7_2_mbox_loc_flat”
bottom: “conv8_2_mbox_loc_flat”
bottom: “conv9_2_mbox_loc_flat”
top: “mbox_loc”
concat_param {
axis: 1
}
}
layer {
name: “mbox_conf”
type: “Concat”
bottom: “conv4_3_norm_mbox_conf_flat”
bottom: “fc7_mbox_conf_flat”
bottom: “conv6_2_mbox_conf_flat”
bottom: “conv7_2_mbox_conf_flat”
bottom: “conv8_2_mbox_conf_flat”
bottom: “conv9_2_mbox_conf_flat”
top: “mbox_conf”
concat_param {
axis: 1
}
}
layer {
name: “mbox_priorbox”
type: “Concat”
bottom: “conv4_3_norm_mbox_priorbox”
bottom: “fc7_mbox_priorbox”
bottom: “conv6_2_mbox_priorbox”
bottom: “conv7_2_mbox_priorbox”
bottom: “conv8_2_mbox_priorbox”
bottom: “conv9_2_mbox_priorbox”
top: “mbox_priorbox”
concat_param {
axis: 2
}
}
layer {
name: “mbox_conf_reshape”
type: “Reshape”
bottom: “mbox_conf”
top: “mbox_conf_reshape”
reshape_param {
shape {
dim: 0
dim: -1
dim: 21
}
}
}
layer {
name: “mbox_conf_softmax”
type: “Softmax”
bottom: “mbox_conf_reshape”
top: “mbox_conf_softmax”
softmax_param {
axis: 2
}
}
layer {
name: “mbox_conf_flatten”
type: “Reshape”
bottom: “mbox_conf_softmax”
top: “mbox_conf_flatten”
reshape_param {
shape {
dim: 0
dim: -1
dim: 1
dim: 1
}
}
}
layer {
name: “detection_out”
type: “DetectionOutput”
bottom: “mbox_loc”
bottom: “mbox_conf_flatten”
bottom: “mbox_priorbox”
top: “detection_out”
include {
phase: TEST
}
detection_output_param {
num_classes: 21
share_location: true
background_label_id: 0
nms_param {
nms_threshold: 0.45
top_k: 400
}
save_output_param {
label_map_file: “data/VOC0712/labelmap_voc.prototxt”
}
code_type: CENTER_SIZE
keep_top_k: 200
confidence_threshold: 0.01
}
}

for (int i = 0; i < inputH * inputW; i++)
    hostInputBuffer[i] = float(fileData[i]);

return true;

}

//!
//! \brief Verifies that the output is correct and prints it
//!
bool SampleMNIST::verifyOutput(const samplesCommon::BufferManager& buffers, const std::string& outputTensorName, int groundTruthDigit) const
{
const float* prob = static_cast<const float*>(buffers.getHostBuffer(outputTensorName));

// Print histogram of the output distribution
gLogInfo << "Output:\n";
float val{0.0f};
int idx{0};
for (unsigned int i = 0; i < 10; i++)
{
    val = std::max(val, prob[i]);
    if (val == prob[i])
        idx = i;
    gLogInfo << i << ": " << std::string(int(std::floor(prob[i] * 10 + 0.5f)), '*') << "\n";
}
gLogInfo << std::endl;

return (idx == groundTruthDigit && val > 0.9f);

}

//!
//! \brief This function uses a caffe parser to create the MNIST Network and marks the
//! output layers
//!
//! \param network Pointer to the network that will be populated with the MNIST network
//!
//! \param builder Pointer to the engine builder
//!
void SampleMNIST::constructNetwork(SampleUniquePtrnvinfer1::IBuilder& builder, SampleUniquePtrnvinfer1::INetworkDefinition& network, SampleUniquePtrnvcaffeparser1::ICaffeParser& parser)
{
const nvcaffeparser1::IBlobNameToTensor* blobNameToTensor = parser->parse(
locateFile(mParams.prototxtFileName, mParams.dataDirs).c_str(),
locateFile(mParams.weightsFileName, mParams.dataDirs).c_str(),
*network,
nvinfer1::DataType::kFLOAT);

for (auto& s : mParams.outputTensorNames)
{
    network->markOutput(*blobNameToTensor->find(s.c_str()));
}

// add mean subtraction to the beginning of the network
Dims inputDims = network->getInput(0)->getDimensions();
mMeanBlob = SampleUniquePtr<nvcaffeparser1::IBinaryProtoBlob>(parser->parseBinaryProto(locateFile(mParams.meanFileName, mParams.dataDirs).c_str()));
Weights meanWeights{DataType::kFLOAT, mMeanBlob->getData(), inputDims.d[1] * inputDims.d[2]};
// For this sample, a large range based on the mean data is chosen and applied to the entire network.
// The preferred method is use scales computed based on a representative data set
// and apply each one individually based on the tensor. The range here is large enough for the
// network, but is chosen for example purposes only.
float maxMean = samplesCommon::getMaxValue(static_cast<const float*>(meanWeights.values), samplesCommon::volume(inputDims));

auto mean = network->addConstant(Dims3(1, inputDims.d[1], inputDims.d[2]), meanWeights);
auto meanSub = network->addElementWise(*network->getInput(0), *mean->getOutput(0), ElementWiseOperation::kSUB);
network->getLayer(0)->setInput(0, *meanSub->getOutput(0));
samplesCommon::setAllTensorScales(network.get(), maxMean, maxMean);

}

//!
//! \brief This function runs the TensorRT inference engine for this sample
//!
//! \details This function is the main execution function of the sample. It allocates
//! the buffer, sets inputs, executes the engine, and verifies the output.
//!
bool SampleMNIST::infer()
{
// Create RAII buffer manager object
samplesCommon::BufferManager buffers(mEngine, mParams.batchSize);

auto context = SampleUniquePtr<nvinfer1::IExecutionContext>(mEngine->createExecutionContext());
if (!context)
{
    return false;
}

// Pick a random digit to try to infer
srand(time(NULL));
const int digit = rand() % 10;

// Read the input data into the managed buffers
// There should be just 1 input tensor
assert(mParams.inputTensorNames.size() == 1);
if (!processInput(buffers, mParams.inputTensorNames[0], digit))
    return false;

// Create CUDA stream for the execution of this inference.
cudaStream_t stream;
CHECK(cudaStreamCreate(&stream));

// Asynchronously copy data from host input buffers to device input buffers
buffers.copyInputToDeviceAsync(stream);

// Asynchronously enqueue the inference work
if (!context->enqueue(mParams.batchSize, buffers.getDeviceBindings().data(), stream, nullptr))
    return false;

// Asynchronously copy data from device output buffers to host output buffers
buffers.copyOutputToHostAsync(stream);

// Wait for the work in the stream to complete
cudaStreamSynchronize(stream);

// Release stream
cudaStreamDestroy(stream);

// Check and print the output of the inference
// There should be just one output tensor
assert(mParams.outputTensorNames.size() == 1);
bool outputCorrect = verifyOutput(buffers, mParams.outputTensorNames[0], digit);

return outputCorrect;

}

//!
//! \brief This function can be used to clean up any state created in the sample class
//!
bool SampleMNIST::teardown()
{
//! Clean up the libprotobuf files as the parsing is complete
//! \note It is not safe to use any other part of the protocol buffers library after
//! ShutdownProtobufLibrary() has been called.
nvcaffeparser1::shutdownProtobufLibrary();
return true;
}

//!
//! \brief This function initializes members of the params struct using the command line args
//!
MNISTSampleParams initializeSampleParams(const samplesCommon::Args& args)
{
MNISTSampleParams params;
if (args.dataDirs.size() != 0) //!< Use the data directory provided by the user
params.dataDirs = args.dataDirs;
else //!< Use default directories if user hasn’t provided directory paths
{
params.dataDirs.push_back(“data/mnist/”);
params.dataDirs.push_back(“data/samples/mnist/”);
}
params.prototxtFileName = “mnist.prototxt”;
params.weightsFileName = “mnist.caffemodel”;
params.meanFileName = “mnist_mean.binaryproto”;
params.inputTensorNames.push_back(“data”);
params.batchSize = 1;
params.outputTensorNames.push_back(“prob”);
params.dlaCore = args.useDLACore;
params.int8 = args.runInInt8;
params.fp16 = args.runInFp16;
params.saveEngine = args.saveEngine;

return params;

}

//!
//! \brief This function prints the help information for running this sample
//!
void printHelpInfo()
{
std::cout << “Usage: ./sample_mnist [-h or --help] [-d or --datadir=] [–useDLACore=]\n”;
std::cout << “–help Display help information\n”;
std::cout << “–datadir Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/mnist/, data/mnist/)” << std::endl;
std::cout << “–useDLACore=N Specify a DLA engine for layers that support DLA. Value can range from 0 to n-1, where n is the number of DLA engines on the platform.” << std::endl;
std::cout << “–int8 Run in Int8 mode.\n”;
std::cout << “–fp16 Run in FP16 mode.\n”;
}

int main(int argc, char** argv)
{
samplesCommon::Args args;
bool argsOK = samplesCommon::parseArgs(args, argc, argv);
if (args.help)
{
printHelpInfo();
return EXIT_SUCCESS;
}
if (!argsOK)
{
gLogError << “Invalid arguments” << std::endl;
printHelpInfo();
return EXIT_FAILURE;
}

auto sampleTest = gLogger.defineTest(gSampleName, argc, const_cast<const char**>(argv));

gLogger.reportTestStart(sampleTest);

MNISTSampleParams params = initializeSampleParams(args);

SampleMNIST sample(params);
gLogInfo << "Building and running a GPU inference engine for MNIST" << std::endl;

if (!sample.build())
{
    return gLogger.reportFail(sampleTest);
}

if (!sample.infer())
{
    return gLogger.reportFail(sampleTest);
}

if (!sample.teardown())
{
    return gLogger.reportFail(sampleTest);
}

return gLogger.reportPass(sampleTest);

}

Hi @GokuDDG ,

The screenshot you have shared points towards the missing entries/declarations in your cpp file.
Please find the below links which will help you to place the specifics in your code.



Thanks!