Poor quality when splitting streams when using nvv4l2decoder

Hi,

I have noticed some strange behaviour when splitting and resizing a stream. In the following pipeline, I am generating a “high-quality” jpeg frame and also a thumbnail quality jpeg frame.

In the first pipeline below, I decode the h.264 stream using nvv4l2decoder ! nvvidconv ! video/x-raw. When doing this the high quality jpeg appears very pixellated. Almost as if it was first scaled to thumbnail quality and then subsequently scaled up again.

GST_DEBUG=3 gst-launch-1.0 filesrc location="Big_Buck_Bunny_1080_10s_30MB.mp4" ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw ! tee name=t ! queue name=q1 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! videoscale ! video/x-raw,width=960,height=540 ! jpegenc ! multifilesink location="/tmp/%d.jpg" t. ! queue name=q2 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! videoscale ! video/x-raw,width=128,height=72 ! jpegenc ! multifilesink location="/tmp/%d_thumb.jpg"

Conversely in this second pipeline I have substituted the decoder with the following omxh264dec ! video/x-raw and in this case both the high-quality and thumbnail images appear as expected with no pixellation.

GST_DEBUG=3 gst-launch-1.0 filesrc location="Big_Buck_Bunny_1080_10s_30MB.mp4" ! qtdemux ! h264parse ! omxh264dec ! video/x-raw ! tee name=t ! queue name=q1 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! videoscale ! video/x-raw,width=960,height=540 ! jpegenc ! multifilesink location="/tmp/%d.jpg" t. ! queue name=q2 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! videoscale ! video/x-raw,width=128,height=72 ! jpegenc ! multifilesink location="/tmp/%d_thumb.jpg"

In both cases the buffer is a video/x-raw before being passed into the tee element so I would have expected both pipelines to have the same result. Any help would be most appreciated.

Sample Video
[1] https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_30MB.mp4

Regards,

I did more investigation here and the following pipeline does work to produce both high resolution and thumbnail quality jpegs without pixellation.

GST_DEBUG=3 gst-launch-1.0 filesrc location="Big_Buck_Bunny_1080_10s_30MB.mp4" ! qtdemux ! h264parse ! nvv4l2decoder ! "video/x-raw(memory:NVMM)" ! tee name=t ! queue name=q1 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! nvvidconv ! "video/x-raw(memory:NVMM),width=960,height=540" ! nvjpegenc ! multifilesink location="/tmp/%d.jpg" t. ! queue name=q2 max-size-buffers=10 max-size-bytes=0 max-size-time=0 silent=true ! nvvidconv ! "video/x-raw(memory:NVMM),width=128,height=72" ! nvjpegenc ! multifilesink location="/tmp/%d_thumb.jpg"

The only fundamental difference here is that I do not use nvvidconv to convert the buffer from video/x-raw(memory:NVMM) to video/x-raw before passing it to the tee element. After that point, I have maintained the video data as video/x-raw(memory:NVMM).

I still maintain that there is something awry about the original pipeline however for future readers (probably myself in a few weeks/months) the pipeline above is a possible workaround.

Regards,

Hi,
If you use jpegenc(software encoder), you may configure the properties and try again:

  quality             : Quality of encoding
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Integer. Range: 0 - 100 Default: 85
  idct-method         : The IDCT algorithm to use
                        flags: readable, writable
                        Enum "GstIDCTMethod" Default: 1, "ifast"
                           (0): islow            - Slow but accurate integer algorithm
                           (1): ifast            - Faster, less accurate integer method
                           (2): float            - Floating-point: accurate, fast on fast HW

Hi @DaneLLL,

Perhaps there is a misunderstanding. The quality seems to be proportional to the width and height of the second part of the pipeline (q2). I don’t believe there is anything wrong with the quality of the image being produced by the jpeg encoder so changing the idct-method isn’t necessary.

Concretely if I split both parts of the “tee” into separate pipelines, both of them will produce nice looking jpegs. It is only when combined as above that you will see the quality degradation. It is really quite striking especially if you make the width and height (currently 128x72) very small.

To be clear, the degradation of quality is seen on the large jpeg not the small jpeg with the “thumb” suffix.

Regards,

hi,
Please use hardware converter nvvidconv and try the property:

  interpolation-method: Set interpolation methods
                        flags: readable, writable, controllable
                        Enum "GstInterpolationMethod" Default: 0, "Nearest"
                           (0): Nearest          - Nearest
                           (1): Bilinear         - Bilinear
                           (2): 5-Tap            - 5-Tap
                           (3): 10-Tap           - 10-Tap
                           (4): Smart            - Smart, this is default
                           (5): Nicest           - Nicest

See if you can get acceptable quality by tuning the mode.

Hi @DaneLLL,

Can you confirm if you have tried the pipeline in the first post?

That is the only pipeline that appears to exhibit the issue. In that pipeline nvvidconv is not being used to scale the frame so there is no reason to change the interpolation method. I understand that it is perhaps not the “most efficient” pipeline.

Regards,

Hi,
You need to specify width and height in source pad of nvvidconv:

gst-launch-1.0 filesrc location="Big_Buck_Bunny_1080_10s_30MB.mp4" ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,width=1920,height=1080,format=I420

Without any setting it takes downstream setting, either 960x540 or 128x72. The issue is that it takes 128x72.

Oh I see! Thanks so much @DaneLLL.