How to make omxh265enc output frame aligned?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

I suspect that ‘omxh265enc’ outputs a gstbuffer that may not be frame aligned.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw,format=(string)I420,width=1280,height=720,
framerate=(fraction)25/1' ! omxh265enc iframeinterval=25 !
"video/x-h265, stream-format=(string)byte-stream, alignment=au" ! omxh265dec ! nvoverlaysink

Above pipeline doesn’t work without h265parse.

What I understood: omxh265enc may output a gst-buffer which contains some data belong to one frame and some data belong to next frame.

How can I force ‘omxh265enc’ to produce only frame align data?

Please suggest if any change is required in gstomx1_src.

2

Hi,
Not sure how to do the implementation, but the code should be in gst_omx_h265_enc_handle_output_frame() or gst_omx_video_enc_handle_output_frame(). You may modify it and rebuild/replace libgstomx.so.

3

Still trying to make it frame aligned.
So far I have compared the buffer before and after ‘h265parse’ plugin. Of-course h265parse is placed before omxh265dec.

raw- frame0: [0, 0, 0, 1, 2, 1, 208, 167, 60, 234, 176, 165, 160, 246, 179, 158, 241, 169, 251, …]
raw- frame1: [0, 0, 0, 1, 2, 1, 208, 175, 47, 230, 70, 171, 70, 183, 153, 248, 232, 122, 41, …]
raw- frame2: [0, 0, 0, 1, 2, 1, 208, 183, 37, 192, 223, 194, 61, 248, 193, 44, 114, 24, 74, …]
raw- frame3: [0, 0, 0, 1, 2, 1, 208, 191, 37, 192, 172, 4, 236, 54, 37, 255, 11, 201, 74, …]

parsed-frame0: [0, 0, 0, 1, 64, 1, 12, 1, 255, 255, 1, 64, 0, 0, 3, 0, 0, 3, 0, 0, 3, 0, 0, …]
parsed-frame1: [0, 0, 0, 1, 2, 1, 208, 15, 36, 192, 248, 45, 21, 14, 229, 216, 1, 134, 182, …]
parsed-frame2: [0, 0, 0, 1, 2, 1, 208, 23, 36, 192, 248, 47, 222, 235, 32, 56, 228, 176, …]
parsed-frame3: [0, 0, 0, 1, 2, 1, 208, 31, 36, 192, 235, 192, 74, 124, 156, 63, 255,…]

Full buffer data is attached below.

But I couldn’t able to figure out, how to make output of omxh265enc completely frame-aligned.

On the contrary output of vaapih265enc is completely frame aligned. I mean buffer before and after h265parse is exactly same.

Raw vs Parsed frames.txt (673 KB)

4

Hi,
The GstBuffer is allocated in

GST_DEBUG_OBJECT (self, "Handling output data");

if (buf->omx_buf->nFilledLen > 0) {
  <b>outbuf = gst_buffer_new_and_alloc (buf->omx_buf->nFilledLen);</b>

You may check if aligning the size here helps.

5

I found this code, but didn’t understand what “aligning the size” means!

6

Hi,
Per
https://devtalk.nvidia.com/default/topic/1066730/jetson-agx-xavier/how-to-replace-vaapih265enc-with-omxh265enc-/post/5402504/#5402504

You say vaapih265enc outputs in aligned buffer size. Do you know it is 4-byte alignment or 8-byte-alignment?

Also you may try x265enc and avdec_h265 plugins. Not sure but probably vaapidecode is specific to work with vaapi encoders.

7

I tried below pipelines and found them functioning well without h265parse plugin.

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=176,height=144' ! videoconvert ! x265enc ! avdec_h265 ! videoconvert ! xvimagesink

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=176,height=144' ! videoconvert ! x265enc ! vaapih265dec ! vaapisink

And I have also checked the buffer data of vaapih265enc before and after h265parse. I found the data exactly same before and after h265parse plugin. I checked this for ~60 frames in a 30fps streaming.

On the contrary, in case of omxh265enc buffer data differs after h265parse plugin.

I dont know, whether vaapih265enc output is 4-byte alignment or 8-byte-alignment.
If you want, I can share some of its output frame data.

8

Hi,
It is working for avdec_h265 to decode compressed data from omxh265enc. The following pipeline can be successfully run:

$ gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! omxh265enc ! 'video/x-h265,stream-format=byte-stream' ! avdec_h265 ! xvimagesink

We don’t have much experience of using vaapi plugins. Need other users to share further suggestion.

9

Yes this pipeline is working.
avdec_h265 is CPU based, so has its overhead.
Strange is omxh265dec is not working.
Once my Nano comes, I will install latest L4T. And will check with omxh265enc-nvv4l2h265dec & nvv4l2h265enc-nvv4l2h265dec.

Intention is hardware h265 codec without h265parse.