Nvv4l2decoder artifacts

Autonomous Machines Jetson & Embedded Systems Jetson TX2
,
1

bein-input.zip (88.3 MB)

Hi.
My release is

# R32 (release), REVISION: 7.2, GCID: 30192233, BOARD: t186ref, EABI: aarch64, DATE: Sun Apr 17 09:53:50 UTC 2022

nvv4l2decoder decodes suggested sample with artifacts

commands:

gst-launch-1.0 -vv filesrc location=bein-input.ts ! tsdemux ! h264parse ! filesink location=bein-input.h264

/usr/src/jetson_multimedia_api/samples/00_video_decode/video_decode H264 --disable-rendering -o bein-input.yuv bein-input.h264

At the bottom of the decoded images you can see an artifact - a strip of damaged pixels

gstreamer pipeline outputs a video with artifacts too:

gst-launch-1.0 filesrc location=bein-input.ts ! tsdemux ! h264parse ! nvv4l2decoder ! nvvidconv  ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvv4l2h264enc ! h264parse ! qtmux! filesink location=bein-input.mp4

looks like a bug

3

Hi,
Please try software decoder to make sure the video content is good:

$ gst-launch-1.0 filesrc location=bein-input.ts ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink
4

Hi.

gst-launch-1.0 filesrc location=bein-input.ts ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink

This command produce images with artifact too: a strip of damaged pixels at the bottom.

Is there something wrong with deinterlacing?

Is there any way to fix this?

5

Hi,
It is more like the video content in the file is corrupted. We would suggest check this. If the video content is valid, we would expect it is correctly decoded through software decoder.

6

Hi.
however, here is this command

gst-launch-1.0 filesrc location=bein-input.ts ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! deinterlace ! videoconvert ! xvimagesink

produce images WITHOUT artifacts

Is there something wrong with deinterlacing in nvv4l2decoder?

8

Hi.

Please clarify. Is this a nvv4l2decoder bug?

9

Hi,
We will check and update. Do you have plan to use Xavier or Orin? Jetpack 4 is going to EoL:
JetPack 4 Reaches End of Life

10

Hi,
On r32.7.2, please use nvvidconv plugin to crop out the region as a solution:

$ DISPLAY=:0 gst-launch-1.0 filesrc location= bein-input.ts ! tsdemux ! h264parse ! nvv4l2decoder ! nvvidconv left=0 right=1920 top=0 bottom=1072 ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! nv3dsink
11

Hi.
Thanks for your solution.
But still, is there a nvv4l2decoder bug or not? I need to know for sure, please.

If a bug exists, can we expect a fix for it on r32.7.2?

12

Hi,
We would need to check further to get more information. The frame data should be 1920x1080 YUV420 but somehow the bottom 8 lines are invalid.

The issue should be specific to decoding the video file. On r32.7.2, please crop the bottom lines out as a solution.

13

Hi.

Is this what you’re talking about?

NVMEDIA: NvMediaMixerInit: 119: frameWidth = 1920, frameHeight = 1088

This is shown in the console when trying to decode a suggested file.

I have examples (interlaced h264) with the same frameHeight = 1088, which are decoded correctly.

I’ll leave the topic open for now, maybe you can say something based on the results of your research, please.

14

Hi,
The normal flow is:

decoded 1920x544 YUV420 data  in top field and bottom filed -> deinterlace into 1920x1088 YUV420 data -> crop into 1920x1080 YUV420 data

Somehow the video file still has invalid lines in bottom and we have to crop one more time into 1920x1072 YUV420 data

15

Hi.

Suggested video have SPS with

 frame_cropping_flag :1
   frame_crop_left_offset :0
   frame_crop_right_offset :0
   frame_crop_top_offset :0
   frame_crop_bottom_offset :2

Perhaps this has implications for proper decoding?

16

Hi,
Do you mean if frame_crop_bottom_offset :2, we should crop 16 bottom lines instead of 8 lines?

17

Hi.

No.

I see in SPS

pic_width_in_mbs_minus1 :119
pic_height_in_map_units_minus1 :33

This means pic size 1920x544 (macroblock 16x16 ). Ok?

And flow like this:

  1. 2 fields decoded 1920x544 YUV420 data (top and bottom)
  2. crop 2 bottom lines into 1920x542 top field
    crop 2 bottom lines into 1920x542 bottom field
  3. deinterlace into 1920x1084 YUV420 data
  4. crop into 1920x1080 YUV420 data

At the end we have (8 - 2*frame_crop_bottom_offset) cropped lines

18

Hi,
It seems like the flow is valid for normal video file since it crops bottom 8 lines. But we see bottom 16 lines are corrupted in this video file. Maybe the video file does not have correct codec data?

19

Indeed, the data may be damaged. I don’t see exactly 16 broken bottom lines, more like 1 or 2.

Fine. But why does the software decoder handle this file better?

20

Hi,
We will compare with good video file and see if there is further finding. And it would be great if you see deviation between the files and share to us.

21

Hi.
No updates?

23

Hi,
We are checking with our team and see if this case can be covered in next Jetpack 4.6.5.