Encoder Produces Artifacts on Scene Change

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
,
1

Dear Nvidia Support,

Please help us to find a solution for an issue with TX2/Jetson Xavier NX/… encoders where the output streams have visible artifacts and defects.

pixelation

The problem manifests itself at the moments when the scene changes; you can see the fragments of the previous scene in the new frames.
We collected video samples in mpeg2video codec and the ways to reproduce the issue.

The Board and Jetpack SDK version is the following:
R32 (release), REVISION: 7.2, GCID: 30192233, BOARD: t186ref, EABI: aarch64, DATE: Sun Apr 17 09:53:50 UTC 2022

We also observed the same behaviour on Jetson Xavier NX hardware.

First, you can try the following gstreamer pipeline that uses nvv4l2decoder and nvv4l2h264enc. In the ouptut you can clearly see the pixelation artifact on the white background after the scene changes.

gst-launch-1.0 -vv filesrc location=~/CNN_sample0.ts ! tsparse ! video/mpegts ! tsdemux ! queue ! mpegvideoparse ! nvv4l2decoder ! queue ! nvv4l2h264enc bitrate=5000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=cnn_jetson01_01.mkv -e
gst-launch-1.0 -vv filesrc location=~/CNN_sample0.ts ! tsparse ! video/mpegts ! tsdemux ! queue ! mpegvideoparse ! nvv4l2decoder num-extra-surfaces=4 ! queue ! nvvidconv  ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvv4l2h264enc bitrate=5000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=cnn_jetson01.mkv -e

To rule out possible issues with gstreamer, we reproduced the same issue with Nvidia’s ./multivideo_transcode sample. It shows the same artifacts in the same position of the video file.

/usr/src/jetson_multimedia_api/samples/16_multivideo_transcode# ./multivideo_transcode num_files 1 ~/CNN_sample0.mpeg2 MPEG2 ~/cnn_mu.h264 H264 --insert-spspps-idr --insert-vui

This is the patch for multivideo_transcode to allow it to ingest mpeg2video encoded streams:

diff -ruN 16_multivideo_transcode/multivideo_transcode_csvparser.cpp 16_multivideo_transcode1/multivideo_transcode_csvparser.cpp
--- 16_multivideo_transcode/multivideo_transcode_csvparser.cpp  2022-04-17 12:54:50.000000000 +0300
+++ 16_multivideo_transcode1/multivideo_transcode_csvparser.cpp 2022-10-13 18:17:52.586856259 +0300
@@ -171,6 +171,10 @@
     {
         return V4L2_PIX_FMT_VP9;
     }
+    if (!strcmp(arg, "MPEG2"))
+    {
+        return V4L2_PIX_FMT_MPEG2;
+    }
     return 0;
 }

diff -ruN 16_multivideo_transcode/multivideo_transcode_main.cpp 16_multivideo_transcode1/multivideo_transcode_main.cpp
--- 16_multivideo_transcode/multivideo_transcode_main.cpp       2022-04-17 12:54:50.000000000 +0300
+++ 16_multivideo_transcode1/multivideo_transcode_main.cpp      2022-10-13 18:17:52.586856259 +0300
@@ -1638,7 +1638,7 @@
         }

         if ((ctx.decoder_pixfmt == V4L2_PIX_FMT_H264) ||
-                (ctx.decoder_pixfmt == V4L2_PIX_FMT_H265))
+                (ctx.decoder_pixfmt == V4L2_PIX_FMT_H265) || ctx.decoder_pixfmt == V4L2_PIX_FMT_MPEG2)
         {
             if (ctx.input_nalu)
             {
@@ -1826,7 +1826,7 @@

         buffer = ctx.dec->output_plane.getNthBuffer(i);
         if ((ctx.decoder_pixfmt == V4L2_PIX_FMT_H264) ||
-                (ctx.decoder_pixfmt == V4L2_PIX_FMT_H265))
+                (ctx.decoder_pixfmt == V4L2_PIX_FMT_H265) || (ctx.decoder_pixfmt == V4L2_PIX_FMT_MPEG2))
         {
             if (ctx.input_nalu)
             {
@@ -2164,4 +2164,4 @@
     }

     return ret;
-}
\ No newline at end of file
+}

Here is the link to the google drive folder with the stream samples:
https://drive.google.com/drive/folders/15DUxwYGxb-s-v9E92jUGqDqlrXAoMQqn?usp=sharing

Thanks,

Ingvar Petrov
Otter Video Inc.

3

Hi,
It seems like decoded YUVs are broken. Please run the command and check if video playback is good:

gst-launch-1.0 -vv filesrc location=~/CNN_sample0.ts ! tsparse ! video/mpegts ! tsdemux ! queue ! mpegvideoparse ! nvv4l2decoder ! nvoverlaysink
4

Hi DaneLLL,

Yes, you are correct, the issue is with the decoder. I run the command and the video playback had issues.

However, when I try decoding this sample with CPU it produces proper output with no artifacts. Also, analyzing the sample I couldn’t find any problems with its structure.

Please advise how could we get a clean picture from the hardware decoder?

Thanks,

Ingvar

5

Hi,
It looks to be MPEG2 stream. Hardware decoder cannot decode all kinds of MPEG2 stream. Please use software decoder for this kind of stream.

6

Hi DaneLLL,

Using a software decoder would significantly decrease the transcoding density of the Jetson module.
Mpeg2video is still a very widely used codec - all TV broadcasts in the US and Canada are coming in MPEG2; and this particular stream was taken directly from b2b stream coming from a sat link of a reputable broadcaster.

Hardware decoder Nvdec is quite capable of decoding this stream with a minor and very annoying exception when there are pixelation artifacts on the screen.

Would you be able to escalate this case to your developers and see if they can try to fix it? If needed, we can collect more samples with this behaviour.

Thanks,

–ip

7

Hi,
We may not be able to handle this in software since it is constraint of hardware decoder. Certain MPEG2 stream can be decoded but certain cannot.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.