TensorRT 4.0 Modification to input.

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Lets say we have a 3x3 matrix as follows:

1    2    3
4    5    6
7    8    9

To actually get a 3x3 matrix as an output, something like this is done:

1    2    3    -FM
4    5    6    -FM
7    8    9    -FM
-FM  -FM  -FM  -FM

where FM = FLOAT_MAX

Here is the starter code.

nvinfer1::ITensor* ninput = input;
nvinfer1::Dims d = input->getDimensions();
nvinfer1::Dims d3 = {3,
                    {d.d[0], d.d[1]+1, d.d[2]+1},
                    {nvinfer1::DimensionType::kCHANNEL, nvinfer1::DimensionType::kSPATIAL,
                     nvinfer1::DimensionType::kSPATIAL}};
ninput->setDimensions(d3);

input contains the (CHW) format tensor.
ninput should contain the (C(H+1)(W+1)) tensor.

Cannot use padding as it will pad from all the sides.

2

Hello,

If you are asking for a TensorRT layer that does this, then Padding layer + Elementwise layer should work.

My colleague recommended

3

Also, just a reminder that DimensionType is deprecated and it is not used for anything inside TRT now.

4

One problem in the approach I think is that it doesn’t modify the dimensions of ITensor* input.

So, I was developing on full defined network and the desired output from maxpool was coming after the stride was applied. Now what I want is that if stride for the layer is 1: then the output size from maxpool should be same.

Right now when the input goes like -
15x15x3 → (2x2,1) → 14x14x3

What is needed (Using padding on one side only)-
15x15x3 → (2x2,1) → 15x15x3

Padding layer + Elementwise layer will not change the output ITensor according to above problem.

Is there any way to access the values of ITensor* so that even while making the new ITensor* the values can be adjusted. If no values are accessible, then what else solution can be thought of ?

nvinfer1::ILayer* netAddMaxpool(int layerIdx, std::map<std::string, std::string>& block,
                                nvinfer1::ITensor* input, nvinfer1::INetworkDefinition* network)
{
    assert(block.at("type") == "maxpool");
    assert(block.find("size") != block.end());
    assert(block.find("stride") != block.end());

    int size = std::stoi(block.at("size"));
    int stride = std::stoi(block.at("stride"));

    nvinfer1::IPoolingLayer* pool
        = network->addPooling(*input, nvinfer1::PoolingType::kMAX, nvinfer1::DimsHW{size, size});
    assert(pool);
    std::string maxpoolLayerName = "maxpool_" + std::to_string(layerIdx);
    pool->setStride(nvinfer1::DimsHW{stride, stride});
    pool->setName(maxpoolLayerName.c_str());

    return pool;
}
5

The IPaddingLayer supports asymmetric padding. You need something like:

// add zero padding to most right column and bottom row
    IPaddingLayer* pad = network->addPadding(*input, DimsHW{0, 0}, DimsHW{1, 1});
    ASSERT(pad);
    
    Weights const_weights;
    
    // code to fill const_weights with zeros and to put -FM values at the padding areas
    // make sure the tensor dimensions match the dimensions of the output tensor of the 'pad' layer
    // ...
    
    IConstantLayer* constant = network->addConstant(pad->getOutput(0)->getDimensions(), const_weights);
    ASSERT(constant);

    IElementWiseLayer* add = network->addElementWise(*constant->getOutput(0), *pad->getOutput(0), ElementWiseOperation::kSUM);
    ASSERT(add);

This will add zero padding to the end of the two most inner dimensions (what TRT calls post padding) of the input tensor, then it will sum the zero padded tensor with a constant tensor filled with zeros and -FM values on the padding areas.