# How to set the correct config for a pytorch model in nvinfer?

**URL:** <https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633>\
**Category:** DeepStream SDK\
**Created:** [April 9, 2021, 2:56am UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633 "2021-04-09T02:56:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![marmikshah](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@marmikshah](https://forums.developer.nvidia.com/u/marmikshah)\
**Post date:** [April 9, 2021, 2:56am UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633/1 "2021-04-09T02:56:26Z")

</div>

Hello,  
I have trained a Resnet classifier in PyTorch and exported to an ONNX model.

The FC layer of Resnet18 is set to:

```auto
model.fc = nn.Sequential(
        nn.Linear(model.fc.in_features, args.n_classes, bias=False),
        nn.Softmax(1),
    )

```

These are the normalization values I have used.

```auto
normalize = transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225])

```

The model accuracy goes up to 80%, but when I infer (using nvinfer) on the exact same images used in training, the result is very different.

I have used the following Offsets & scale factor

```auto

# RGB, torchvision = 255*[0.485;0.456;0.406]
offsets=123.675;116.28;103.53

maintain-aspect-ratio=1

#net-scale-factor=0.003921569
net-scale-factor=0.01735207357

```

With that in mind, I wanted to know

1. How to set the correct offsets and net-scale-factor to match my training normalization values?
2. Is there any place where I can find some examples of how nvinfer does the asymmetric padding when `maintain-aspect-ratio=1`
3. Is there any official documentation to use a pytorch model as an SGIE in Deepstream?

Thanks.

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [April 9, 2021, 3:47am UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633/3 "2021-04-09T03:47:01Z")

</div>

Hi,

**1**

The preprocessing equation used in the pyTorch is `y = (x - mean) / std` .  
In deepstream, it is `y' = net-scale-factor * ( x' - mean')`.  
So please set the `mean'=mean`, and `net_scale_factor=1/std`.

One problem is that we don’t support channel-wise normalization.  
You can either use the average _std_ value or update the source code below:

> /opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer\_context\_impl.cpp

**2.** Do you mean the asymmetric padding in the network architecture, like conv?  
If yes, the network definition should be identical to the training frameworks.

**3.** Below is a sample for ONNX model.  
Although it is used as PGIE, you can follow the same for SGIE.

> <https://github.com/NVIDIA-AI-IOT/deepstream_pose_estimation/blob/master/deepstream_pose_estimation_config.txt>

Thanks.

---

<div class="post-metadata">

**Author:** ![marmikshah](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@marmikshah](https://forums.developer.nvidia.com/u/marmikshah)\
**Post date:** [April 9, 2021, 4:16am UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633/4 "2021-04-09T04:16:43Z")

</div>

Hi Aasta,

Thank you for your quick reply. I will take a look at point 1 and 3.

For asymmetric padding I want to know what kind of input is being sent to the model.  
In the documentation of nvinfer’s properties it is mentioned “Indicates whether to maintain aspect ratio while scaling input. DeepStream currently does asymmetric padding only.” for `maintain-aspect-ratio`

Does that mean that there is another layer added which does the padding? Or is it like a pre-processing step done in nvinfer before the object is sent to the model? If so, is there any way to see how the image looks after the padding is added.  
I simply want to verify whether the input image to nvinfer (after padding) matches my preprocessing when training in Pytorch.

Thanks

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [April 19, 2021, 9:08am UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633/5 "2021-04-19T09:08:28Z")

</div>

Hi,

There are two different padding available.

For Deepstream, there is a padding that try to feed input data into network buffer.  
In this usecase, the related to configure file is `maintain-aspect-ratio`.

There is also a layer-level padding which is implemented by TensorRT.  
You can find some information below:

> **[Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#padding-layer)**
>
> This is the revision history of the NVIDIA TensorRT 8.4 Developer Guide.

Thanks.

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [October 12, 2021, 4:14pm UTC](https://forums.developer.nvidia.com/t/how-to-set-the-correct-config-for-a-pytorch-model-in-nvinfer/174633/7 "2021-10-12T16:14:28Z")

</div>

Closed on October 12, 2021, 4:14pm UTC