Encoding with nvv4l2h265enc, using nvcompositor and nvarguscamerasrc

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
1

Hi,

I am developing with the following device:

  • Jetson Orin NX
  • GStreamer 1.16
  • Jetpack R35.5.0

In the device I am trying to make it work the following GStreamer pipeline:

gst-launch-1.0 nvcompositor name=comp \
          sink_0::width=3040 \
          sink_1::width=3040  \
          sink_1::xpos=3040 ! \
     "video/x-raw(memory:NVMM),width=(int)6080,height=(int)4032,format=(string)RGBA,framerate=21/1" ! \
     nvvidconv ! \
     "video/x-raw(memory:NVMM),width=(int)6080,height=(int)4032,format=(string)NV12,framerate=21/1" ! \
     nvv4l2h265enc iframeinterval=21 insert-sps-pps=1 insert-vui=1 num-B-Frames=0 maxperf-enable=1 preset-level=3 bitrate=8000000 ! \
     queue ! \
     h265parse ! \
     rtspclientsink latency=0 protocols=tcp location='rtsp://<IP>:8554/testfluendo' \
     nvarguscamerasrc sensor-id=0 tnr-mode=2 tnr-strength=0.75 ee-mode=2 ee-strength=0 ! \
     'video/x-raw(memory:NVMM), width=4032, height=3040, format=(string)NV12, framerate=(fraction)21/1' ! \
     videorate silent=false ! \
     nvvidconv flip-method=1 ! \
     comp. \
     nvarguscamerasrc sensor-id=1 tnr-mode=2 tnr-strength=0.75 ee-mode=2 ee-strength=0 ! \
     'video/x-raw(memory:NVMM), width=4032, height=3040, format=(string)NV12, framerate=(fraction)21/1' ! \
     videorate silent=false ! \
     nvvidconv flip-method=3 ! \
     comp.

Using nvv4l2h264enc and h264parse it works fine for lower resolutions, but nvv4l2h265enc makes it fail at any resolution. My final goal it’s to encode with H265, using the nvcompositor, and to keep the memory as NVMM all the time. The desired output resolution is 6080x4032.

Using GStreamer 1.16 makes nvcompositor to fails even with nvv4l2h264enc. However, using GStreamer 1.24 with the pipeline works for nvv4l2h264enc.

I can see the following traces before failing:

GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 1 
   Camera mode  = 0 
   Output Stream W = 4032 H = 3040 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Available Sensor modes :
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: 4032 x 3040 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 4032 H = 3040 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
NvVideo: H265 : Profile : 1 
NVMEDIA: Need to set EMC bandwidth : 2010400 

(gst-launch-1.0:9626): GStreamer-CRITICAL **: 20:11:32.653: gst_buffer_fill: assertion 'src != NULL || size == 0' failed

(gst-launch-1.0:9626): GStreamer-CRITICAL **: 20:11:32.653: gst_buffer_fill: assertion 'src != NULL || size == 0' failed
*** stack smashing detected ***: terminated
Aborted (core dumped)

I even compiled the jetson_36.4 versions of nvarguscamerasrc and nvcompositor, without updating the whole Jetpack, but it didn’t change anything either.

Can you help me find out why:

  • the nvv4l2h265enc fails
  • to get the lowest latency for that setup
4

Hi,
Hardware encoder supports 4K as maximum resolution. Please downscale to 4K and send to hardware encoder.

The capability is stated in

Jetson Download Center | NVIDIA Developer

5

I see. I am wondering why with another pipeline using videomixer instead of the nvcompositor, it seems the nvv4l2h265enc is able to handle that resolution for that case?

/GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, width=(int)6080, height=(int)3040, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)21/1, interlace-mode=(string)progressive, colorimetry=(string)bt2020, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)6080, height=(int)3040, framerate=(fraction)21/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive

I could share the whole pipeline in a private way if needed

7

Hi,
We would suggest follow the capability in module data sheet to develop your use-case. System stability may be impacted if you run the use-case above the spec.

If you would like to explore further, you may check source code of nvcmopositor plugin:

Jetson Linux | NVIDIA Developer
Driver Package (BSP) Sources

8

Thank you so much @DaneLLL , we will forward this message to our client.