DeepStream TRT preprocessing settings

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I don’t understand how to adopt std and mean preprocessing to DS config: offsets and net-scale-factor.

Accroding to PyTorch normalization in Deepstream config and Image preprocess question - #7 by AastaLLL, there are option to calculate average net-scale-factor for std, therefore we use average value 0.226 from std = [0.229, 0.224, 0.225], and get the following value for net-scale-factor=1/128/0.578* 0.226 = 0.0030547145328

  1. How to adopt mean values mean= [0.485, 0.456, 0.406] to DS config??
  2. Can I edit preprocessing calculations in nvinfer sources?
3

Hi,

1.
The preprocess formula is y = net-scale-factor*(x-offsets).
Since we don’t support channel-based scaling value, you can approximate the net-scale-factor for simplicity like this:

mean=(0.485+0.456+0.406)/3
net-scale-factor = 2/255*mean

2.
Yes. You can find the pre-processing source in the below file:
/opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_conversion.cu

Ex.

__global__ void
NvDsInferConvert_CxToP3FloatKernel(
    float *outBuffer,
    unsigned char *inBuffer,
    unsigned int width,
    unsigned int height,
    unsigned int pitch,
    unsigned int inputPixelSize,
    float scaleFactor)
{
    unsigned int row = blockIdx.y * blockDim.y + threadIdx.y;
    unsigned int col = blockIdx.x * blockDim.x + threadIdx.x;

    if (col < width && row < height)
    {
        for (unsigned int k = 0; k < 3; k++)
        {
            outBuffer[width * height * k + row * width + col] =
                scaleFactor * inBuffer[row * pitch + col * inputPixelSize + k];
        }
    }
}

Thanks.