Accuracy issues while running a model inside and outside deepstream with same trt modeld

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am using a densenet161 trt model with fp16 format. When I run that model outside the deepstream with the trtmodel, I was able to get the same accuracy as like fp32. But when I place in inside the deepstream the scores are difference. How to understand whats going on ?

I am suspecting that difference is happening at the preprocessing step, but not sure.

python version:
self.mean, self.std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]
self.trans = transforms.Compose(
[
transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
]
)

Deepstream version:
gpu-id=0
model-engine-file=…/ds_models/stable/densenet161_4x3x224x224_ls_fp16.trt
batch-size=4
net-scale-factor=0.01735207357
scaling-filter=1
offsets=123.675;116.28;103.53
force-implicit-batch-dim=1
model-color-format=0
network-mode=2
process-mode=2
network-type=100
output-tensor-meta=1
gie-unique-id=3
operate-on-gie-id=1
operate-on-class-ids=0
interval=0
output-blob-names=score

To get the scores in deepstream,

if user_meta and user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META:
    try:
        tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
    except StopIteration:
        break

    # Check if we took the correct GIE-UNIQUE-ID
    if tensor_meta.unique_id != self.UNIQUE_ID:
        try:
            l_user = l_user.next
        except StopIteration:
            break
        continue
    layer = pyds.get_nvds_LayerInfo(tensor_meta, 0)
    score = pyds.get_detections(layer.buffer, 0)

Questions:

  1. Is the net-scale-factor and the offsets correct compared with mean and std?
  2. How can I debug to check where the issue is happening with the deepstream version?
  3. How to access the input of the model (before and after preprocessing step)?

Please let me know how I can fix this issue.

Thank you,

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only) 5.1.2
• TensorRT Version 8.5.2
• Issue Type( questions, new requirements, bugs) bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

3

You need to set that according to our formula below.

y = net-scale-factor*(x-mean)
x is the input pixel value. It is an int8 with range [0,255].
mean is the corresponding mean value, read either from the mean file or as offsets[c], where c is the channel to which the input pixel belongs, and offsets is the array specified in the configuration file. It is a float.
net-scale-factor is the pixel scaling factor specified in the configuration file. It is a float.
y is the corresponding output pixel value. It is a float.

You can refer to our FAQ to tune some parameter first.

You can refer to our FAQ.

4

Firstly, thanks for the quick response. Looks like the net-scale-factor and offsets is exactly same if the input is scaled with LINEAR interpolation,

x_orig_resize = cv2.resize(x_orig.copy(), (self.height, self.width) , interpolation=cv2.INTER_LINEAR)

Here in the deepstream config, I dont find linear interpolation operation,
https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/DeepStream_Development_Guide/baggage/group__ee__buf__surf__transform.html#ga29da322f0c4f4ed65d19cc70471f54c0:~:text=enum%20NvBufSurfTransform_Inter

NvBufSurfTransformInter_Nearest --> Specifies Nearest Interpolation Method interpolation.
NvBufSurfTransformInter_Bilinear --> Specifies Bilinear Interpolation Method interpolation.
NvBufSurfTransformInter_Algo1 --> Specifies GPU-Cubic, VIC-5 Tap interpolation.
NvBufSurfTransformInter_Algo2 --> Specifies GPU-Super, VIC-10 Tap interpolation.
NvBufSurfTransformInter_Algo3 --> Specifies GPU-Lanzos, VIC-Smart interpolation.
NvBufSurfTransformInter_Algo4 --> Specifies GPU-Ignored, VIC-Nicest interpolation.
NvBufSurfTransformInter_Default --> Specifies GPU-Nearest, VIC-Nearest interpolation.

Any quick suggestion on how to set the linear interpolation in the config with scaling-filter please?

Thank you,

5

You can refer to the Guide of OpenCV. The INTER_LINEAR is bilinear interpolation.

6

Yes. I tried playing around with those properties, but I still have issues reproducing the same output.

I noticed the difference in the images after processing, comparing three channels of std values in Python and only one std value (mean-of-std), i.e., net-scale factor.

For another model as well, the std I have is
std → [0.2471, 0.2214, 0.2157]
net-scale-factor → [0.01587037, 0.0177126, 0.01818066]

I tried setting the mean of it, but that error is still causing some issues.

Is it possible to use a different std for each channel instead of computing the mean?

7

No. The nvinfer does not support standard deviation currently.
You need to customize your own tensor data with your algorithm through our nvdspreprocess.

8

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.