AV1 encode/decode on Jetson Orin Nano r36.4

Robotics & Edge Computing Jetson Systems Jetson Orin NX
, ,
1

Hi, whatever I do, I get error messages when using accelerated AV1 decode with gstreamer. I’ve made simple back-to-back pipelines to pinpoint the problem and all lead to:

ParseObuFrameHeader: 2367: Invalid buf_idx = -1 or offset

Simplified from AV1/RTSP Streaming Not Working - #25 by DaneLLL :

gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw, format=NV12, width=3840, height=2160, framerate=30/1 \
   ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM)' \
   ! queue ! nvv4l2av1enc insert-seq-hdr=true \
   ! queue ! av1parse \
   ! queue ! nvv4l2decoder ! nvvidconv ! fakevideosink

Another one showing the same symptoms, inspired by Orin NX Gstreamer AV1 Round-Trip (Missing av1parse?) :

gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw,framerate=30/1,width=1920,height=1080 \
   ! nvvidconv ! nvv4l2av1enc ! video/x-av1 \
   ! matroskamux ! queue ! matroskademux \
   ! queue ! nvv4l2decoder ! fakevideosink

When I replace nvv4l2decoder with av1dec, the error magically disappears. Is there an issue with nvv4l2decoder for AV1?

2

*** Please note that this reply is generated by LLM automatically ***

Based on the provided context, it appears that you are experiencing issues with accelerated AV1 decode using GStreamer on Jetson Orin Nano. You have created simple back-to-back pipelines to isolate the problem, and all of them lead to errors.

The error messages are not explicitly provided, but based on the context, it seems that the issue is related to the nvv4l2decoder element. When you replace nvv4l2decoder with av1dec, the error disappears.

This suggests that there might be an issue with the nvv4l2decoder element when it comes to decoding AV1 streams. The nvv4l2decoder element is a hardware-accelerated decoder that uses the NVIDIA V4L2 API to decode video streams. However, it seems that it might not be fully compatible with AV1 streams.

On the other hand, av1dec is a software-based decoder that is specifically designed to decode AV1 streams. It is likely that av1dec is more compatible with AV1 streams than nvv4l2decoder.

It’s worth noting that the context mentions that AV1 encode using gst-v4l2 is only supported on Jetson Orin, but it does not explicitly mention anything about AV1 decode. This might suggest that AV1 decode is not fully supported on Jetson Orin using the gst-v4l2 pipeline.

To resolve the issue, you can try using the av1dec element instead of nvv4l2decoder for decoding AV1 streams. This might require modifying your pipeline to use the av1dec element instead of nvv4l2decoder.

Here is an example pipeline that uses av1dec for decoding AV1 streams:

gst-launch-1.0 filesrc location=<filename.mp4> ! \
    qtdemux name=demux demux.video_0 ! queue ! av1dec ! \
    videoconvert ! autovideosink

This pipeline uses the av1dec element to decode the AV1 stream, and then converts the decoded video to a format that can be displayed using the autovideosink element.

In summary, it appears that there might be an issue with the nvv4l2decoder element when it comes to decoding AV1 streams on Jetson Orin Nano. Using the av1dec element instead of nvv4l2decoder might resolve the issue.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

4

Dear LLM,

I have given the error message in my post but you haven’t spotted it. It is “ParseObuFrameHeader: 2367: Invalid buf_idx = -1 or offset".

The rest of your reasoning is correct, it repeats what I have stated, that replacing nvv4l2decoder with av1dec fixes the issue. You have not caught, probably because it is not well documented, that nvv4l2decoder is a hardware accelerated operation while av1dec executes in software and consumes lots of CPU. I’m looking for a pipeline with hardware acceleration for both encode and decode. Nvidia’s gstreamer documentation leaves me with the impression that this should be easily possible but it doesn’t work as expected. Great if you or a human can come up with help.

5

Hi @umbledore,

I hope you are doing well.

I came across this forum: AV1 encoder disables hardware decoding in Chrome on forceKeyFrame / PLI, which is related to AV1 encode/decode. There is currently an on-going investigation related to av1parse breaking the stream. I am not sure if this is exactly the same issue you are seeing, but it seems related and might be helpful you with your investigation.

Best regards,
Nico
Embedded Software Engineer at ProventusNova

7

Hi,
Orin Nano does not have hardware encoder, so nvv4l2av1enc plugin does not work. Please try software AV1 encoder.

8

Hi Dane, I believe I have an early version of the dev kit, from before Nano/NX naming change. I’m under the strong impression that the board has hardware accelerated AV1 enc/dec.

  1. The card box says “Nano” but it identifies as 3767 “COMPATIBLE_SPEC 3767-000-0000–1–jetson-orin-nano-devkit-” which in my and LLMs understanding has the full set of AV1 accelerators.
  2. When I run the pipeline, NVENC and NVDEC turn on: tegrastats says something like NVENC 73%@371371 NVDEC 80%@230
  3. The nvv4l2av1enc plugin produces a bit stream.
9

Hi Nico, thanks for the link! This could really be caused by the same issue even though it is not easy to see. Probably just need to wait for a resolution from Nvidia.

To your comment, I’m not convinced that it is related to av1parse. Actually not sure what it is supposed to do in this case, just took it from an example. When I run encode-decode back-to-back without av1parse, the same error messages appear.

I probably just need to assume that AV1 is broken for now even though I’m surprised that this has not been caught by an automated regression test in Nvidia. It’s good that they seem to be looking into it. Accelerated AV1 enc and dec is really promising.

1 Like
10

Hi,
The device looks to be Orin NX. Please try the command and check if there is video playback:

DISPLAY=:0 gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw,framerate=30/1,width=1920,height=1080 \
   ! nvvidconv ! nvv4l2av1enc ! video/x-av1 \
   ! matroskamux ! queue ! matroskademux \
   ! queue ! nvv4l2decoder ! nvvidconv ! video/x-raw ! xvimagesink
11

Hi Dane, yes, there is playback but I still see the error messages. Even this simplified pipeline plays (with the error message):

gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw,framerate=30/1,width=3840,height=2160 \
! nvvidconv ! nvv4l2av1enc \
! queue ! nvv4l2decoder ! nvvidconv ! xvimagesink

obu_invalid_idx.log (6.4 KB)

12

Hi,
We will check why the print is triggered. Decoding function should work.

14

Any update?

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.