Output of tensorflow model in deepstream not accurately same to python inference

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform: RTX 2080
• DeepStream Version: 5.0
• NVIDIA GPU Driver Version (valid for GPU only): 440.33.01
• Issue Type( questions, new requirements, bugs)

Hi,
I am testing out a few of my models but there seems to be a difference of few pixels in output which can negatively impact the accuracy of the models and also secondary inference models are very sensitive to the output of primary inference.

I have tried the same with ssd_inception_v2 model provided in deepstream models_repo. Even there I find the raw output(before postprocessing) to be slightly different from the python inference. I have used the sample_720p.jpg to compare the result.
here is the python script-
infer.txt (5.9 KB)
I use config_infer_primary_detector_ssd_inception_v2_coco_2018_01_28.txt for deepstream inference and print the raw inference output in nvdsinfer_custombboxparser.cpp to compare.

Thanks.

3

Hi,

A common issue is from the image pre-processing stage.
Have you applied the similar scaling and normalization to the deepstream?

Thanks.

4

Hi,
I have provided scale_factor=1 and channel_offsets:[0,0,0] and also the maintain_aspect_ratio is set to 0. I will include the config file to compare with python script. There is no obvious difference that I see.
config_infer_primary_detecter_ssd_inception_v2_coco_2018_01_28.txt (1.3 KB)

5

Hi,

Sorry for the late.
Could you help to check if this issue can be reproduced with our standard example below:

Thanks.

7

Hi,

I compared the output from deepstream and python TensorFlow for the FasterRCNN Inception V2 model. I could still find a difference in the raw output. I used sample_720p.jpg to get the results and compare them.

The list of output given by deepstream -
[detection_boxes] detection_classes detection_scores
[ 0.428617 0.15843 0.772679 0.247405 ] 1 0.958776
[ 0.396941 1.96457e-05 0.927068 0.107122 ] 1 0.906904
[ 0 0.371138 0.968965 1 ] 7 0.813362
[ 0.0175899 0.349852 0.994987 1 ] 6 0.786315
[ 0.46349 0.313964 0.528804 0.370895 ] 3 0.567978
[ 0.464487 0.318552 0.530802 0.371105 ] 8 0.479378
[ 0.44496 0.297592 0.477602 0.326381 ] 3 0.390182
[ 0.0893252 0.750784 0.990411 0.990492 ] 82 0.32203
[ 0.441482 0.277032 0.476558 0.304725 ] 3 0.31688
[ 0.450016 0.291476 0.481224 0.318114 ] 3 0.300619

list of given by python TensorFlow script-
[detection_boxes] detection_classes detection_scores
[0.418341 0.1497689 0.76314723 0.24465884] 1.0 0.98760206
[3.69310945e-01 1.06353764e-04 9.30950046e-01 1.08975887e-01] 1.0 0.88125163
[0.46232662 0.31416744 0.52475774 0.36643988] 3.0 0.87001723
[0. 0.35897997 0.9888007 1. ] 7.0 0.7568224
[0. 0.3749511 0.98827183 1. ] 6.0 0.6742431
[0.44705787 0.29565686 0.47802058 0.32768974] 3.0 0.3348193
[0.44225553 0.27591562 0.4755208 0.30208823] 3.0 0.3193855

My deepstream pipeline is
filesrc → jpegparse → nvv4l2decoder → streammux → pgie(nvinferserver) → fakesink

nvinferserver config-file
config_infer_primary_faster_rcnn_inception_v2.txt

9

Hi,

We want to reproduce this in our environment.
Could you share the detailed steps and configuration file with us?

Thanks.

10

Hi,
The following files will help to reproduce deepstream portion
deepstream_test_app.cpp (14.4 KB)
nvdsinfer_custombboxparser.cpp (10.0 KB)
config_infer_primary_faster_rcnn_inception_v2.txt (2.1 KB)

The following file will help in reproducing python results
infer_rcnn.txt (3.9 KB) (the python sript is saved as txt)

Hope this helps recreate the results.
Thanks.

11

Thanks for the information.
We are trying to reproduce this problem internally.

Stay tuned.

13

Hi,

We try to execute the infer_rcnn.py but got a killed error.
Since our GPU has 32G memory, it’s more likely an app issue rather than OOM.

Could you share which TensorFlow version do you use?
Does the script work with TensorFlow v1.15.x?

We noticed that you use a different image decoder in the Deepstream pipeline (Gstreamer) and python script (OpenCV).
So it is possible that the image data has some variation and leads to the inference output has a slight difference.
Could you check this possibility? For example, dump the input image?

Thanks.

14

Hi,
I have tensorflow : 1.14.0 installed in my system but the code is fairly generic and I think it should work with any tensorflow version. And memory won’t be an issue as my system is 12G GPU memory.

As for the decoder, I can dump the values(just before inferencing) for python script but how can I dump raw pixel values for deepstream case to compare those to python script input?
Could you suggest the same?

Thanks.

15

Hi,

Instead of dumping raw pixels from Deepstream, you can feed the OpenCV raw data into the pipeline.
For example:

gst-launch-1.0 multifilesrc location='test_\%d.raw' ! videoparse format=rgb width=224 height=224 ! videoconvert ! 'video/x-raw,format=(string)RGBA, width=224, height=224' !  nvvideoconvert ! 'video/x-raw(memory:NVMM),format=(string)RGBA, width=224, height=224' ! mux.sink_0 nvstreammux live-source=0 name=mux batch-size=1 width=224 height=224 ! nvinfer ...

Thanks.