Small distortion/noise on the frames acquired through get_nvds_buf_surface function

• Jetson Xavier NX
• DeepStream 5.1
• JetPack 4.5.1
• TensorRT
• Issue Type: bugs… and questions
• How to reproduce the issue?

I think the easiest way to reproduce this issue will be to create a dummy image and send it through an RTSP stream and access it inside the deepstream-test3 example, and compare the two images at a pixel level (pretty much, that is exactly what I’m doing, but my setup also includes ROS, and that will be too much of a headache to make a setup just for this).

The problem

For debugging purposes, I’m creating a dummy RGBA image with these values on the Red, Green, Blue and Alpha channels:

[24, 0, 0, 255]

And I’m sending this image through an RTSP stream to DeepStream, in a modified version of the deepstream-test3 example (modified in such a way so that I can access the frames through the get_nvds_buf_surface function, pretty similar with the deepstream-imagedata-multistream) where I am acquiring this image, but here, has other values:

[38, 1, 0, 255]

And this is a big problem for me, because even though visually there is no difference, I want that from these values to reconstruct a 16UC1 depth image (e.g. in this case the resulting depth image would have only these values: [2400]), but in this version it has [3810], so for me, even a +/- 1 difference in a pixel is very important.

Questions and trials

I am pretty new to DeepStream, Gstreamer, RTSP, basically one of my first projects of this kind, so practically,
I know nothing about them. That is why I have a few questions:

  1. Does RTSP applies any kind of compression/transformations upon images that might affect
    the image, or it should leave the image completely untouched?

  2. Might be Gstreamer that somehow distorts the image because of some settings that
    I’ve might set, or not?
    Even though I’ve also tried the deepstream-imagedata-multistream
    example and there was the same result, same “noise”.

  3. Might this small distortion be a result of some operations applied on the image
    (e.g. resize, reshape) during its journey through DeepStream?

  4. Basically, the main question would be: has anyone else seen this kind of “noise”
    happening to an image, compared to what it was before going through
    an RTSP stream and after it was processed by DeepStream?

It is worth mentioning, once again, that these distortion values are not greater
than +/- 15 on a pixel (as far as I could observe), so almost invisible changes to the eye.
For as much as I could found about this topic, I’ve seen a few cases of distorted images,
but those were serious/obvious cases of data alterations, but I couldn’t find anything
about this pixel level distortion.

Below it’s the content of my dstest3_pgie_config.txt:



And my modified version of (18.3 KB)

I would very much appreciate any kind of response/comment/remark/guidance/advice regarding
this problem as I don’t know if it is a problem of the aforementioned tools,
or it is a problem of how I’m using them, because if it is the latter,
I might still have a chance of solving this, otherwise I might be forced to give up
on this idea of accessing depth images inside DeepStream. Thanks!

RTSP can not transferring RGBA data, what is your format of the rtp payload?

It is not reasonable. There are lots of data processing such as scaling, format conversion,… The input and output can not be exactly the same.

1 Like
  1. This question has nothing to do with Nvidia. Please investigate by yourself.
  2. Lots of video/image transform happens inside the pipeline, they are not lossless.
  3. The same to question 2
  4. The output can not be exactly the same as input. Even without deepstream, your RTSP stream is already different to your original images.

Please learn image/video processing knowledge by yourself. It has nothing to do with deepstream.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.