Nvjpegenc issues with high resolution stream

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

Hello,
I faced with a nvjpegenc issue while developing a Gstreamer jpeg encoding pipeline, here is a major part of my pipeline:

nvarguscamerasrc sensor-id=0 !
video/x-raw(memory:NVMM),width=1920,height=1200,framerate=30/1,format=NV12 !
comp.sink_0
nvarguscamerasrc sensor-id=1 !
video/x-raw(memory:NVMM),width=1920,height=1200,framerate=30/1,format=NV12 !
comp.sink_1 nvcompositor name=comp background=0
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1200 sink_0::sizing-policy=1
sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1200 sink_1::sizing-policy=1 !
video/x-raw(memory:NVMM),width=3840,height=1200 ! 
queue !
nvvidconv !
video/x-raw(memory:NVMM),width=3840,height=1200,format=NV12 ! tee name=timg0 !
# something is going on here but it is not related to the issue
timg0. !
nvvidconv ! video/x-raw(memory:NVMM),format=NV12,width=$IMAGEW,height=$IMAGEH,framerate=(fraction)15/1 ! videorate max-rate=15/1 !
queue ! nvjpegenc ! fakesink

My setup is: Jetson Orin AGX, Jetpack 6.2, Jeston Linux 36.4.3
Let me describe the issue:
Image size on the jpeg encoder’s input is configurable and can be 3840x1200 or resized down
When it is resized to the any ratio less than 3840x1200 (1920x600, 3456x1080, whatever else) everything works fine.
However, when the $IMAGEW and $IMAGEH params are set to the same resolution as the input one (3840x1200) I get a segfault:

video_head-1  | [gst_pipeline_video_node-1] Segmentation fault detected! Generating stack trace...
video_head-1  | [gst_pipeline_video_node-1] Stack trace:
video_head-1  | [gst_pipeline_video_node-1] /root/ws/install/lib/libgst_pipeline.so(_ZN12gst_pipeline16segfault_handlerEi+0x6c) [0xffff8fa16344]
video_head-1  | [gst_pipeline_video_node-1] linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff94d0d79c]
video_head-1  | [gst_pipeline_video_node-1] /usr/lib/aarch64-linux-gnu/nvidia/libnvjpeg.so(jpegTegraEncoderAccelerationCheck+0x10c) [0xffff5484f380]
video_head-1  | [gst_pipeline_video_node-1] /usr/lib/aarch64-linux-gnu/nvidia/libnvjpeg.so(jpeg_start_compress+0xfc) [0xffff548158fc]
video_head-1  | [gst_pipeline_video_node-1] /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so(+0x51cc) [0xffff405951cc]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libgstvideo-1.0.so.0(+0x44abc) [0xffff74244abc]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0(+0x94a78) [0xffff8f374a78]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0(+0x97cb8) [0xffff8f377cb8]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0(gst_pad_push+0x124) [0xffff8f3780e8]
video_head-1  | [gst_pipeline_video_node-1] /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstcoreelements.so(+0x436fc) [0xffff414236fc]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0(+0xc180c) [0xffff8f3a180c]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libglib-2.0.so.0(+0x87668) [0xffff8f4d7668]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libglib-2.0.so.0(+0x84990) [0xffff8f4d4990]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libc.so.6(+0x80398) [0xffff93810398]
video_head-1  | [gst_pipeline_video_node-1] /lib/aarch64-linux-gnu/libc.so.6(+0xe9e9c) [0xffff93879e9c]

So as you can see, in this case I literally take 3840x1200 frames from one nvvidconv and put them to another nvvidconv which is set up to the same input resolution, seems like in this case it somehow changes surfaceList[0].dataPtr to NULL and the downstream nvjpegenc fails to compute it. (it is just my rough assumption).
Note, that if I just remove the second nvvidconv from the pipeline it works with no issues, and it also works if I put a 1920x1200 input to the second nvvidconv and convert it to 1920x1200 (no changes). It starts emitting segfaults only when I try to convert 3840x1200 → 3840x1200

Could you please take a look and try to fix this bug please

2

*** Please note that this reply is generated by LLM automatically ***

Based on the provided information, it appears that you are experiencing a segmentation fault when using the nvjpegenc element in your GStreamer pipeline on the Jetson Orin AGX with Jetpack 6.2 and Jetson Linux 3.6.4.3. The issue occurs when you try to encode a 3840x1200 image at the same resolution, but it works fine when resizing the image to a lower resolution.

To troubleshoot this issue, let’s analyze the pipeline and the elements involved.

  1. nvjpegenc element: This element is used for JPEG encoding and is part of the NVIDIA GStreamer plugins. It supports various input formats, including RAW-YUV formats.
  2. nvvidconv element: This element is used for video conversion and scaling. It supports various input formats, including RAW-YUV formats.

Based on the provided context, it seems that the nvvidconv element is being used to convert the input image to a different format before passing it to the nvjpegenc element.

Here are a few potential causes for the segmentation fault:

  1. Memory allocation issue: When the input resolution is the same as the output resolution, the nvvidconv element might not allocate enough memory to store the converted image, leading to a segmentation fault.
  2. Format conversion issue: The nvvidconv element might not be able to handle the format conversion correctly when the input and output resolutions are the same, leading to a segmentation fault.
  3. Buffer management issue: The nvjpegenc element might not be able to manage the buffers correctly when the input resolution is the same as the output resolution, leading to a segmentation fault.

To fix this issue, you can try the following:

  1. Use a different video conversion element: Instead of using nvvidconv, you can try using a different video conversion element, such as videoconvert, to see if the issue persists.
  2. Add a buffer pool: You can try adding a buffer pool element, such as bufferpool, to manage the buffers and see if it resolves the issue.
  3. Check the input format: Make sure that the input format is supported by the nvjpegenc element. You can try using a different input format, such as I420, to see if the issue persists.
  4. Update the NVIDIA GStreamer plugins: Make sure that you are using the latest version of the NVIDIA GStreamer plugins. You can try updating the plugins to see if it resolves the issue.

Here is an example pipeline that you can try:

gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, width=3840, height=1200 ! nvjpegenc ! filesink location=output.jpg

This pipeline uses the videoconvert element to convert the input image to the I420 format, which is supported by the nvjpegenc element.

If none of these suggestions resolve the issue, please provide more information about your pipeline and the error messages you are seeing, and I’ll be happy to help you further.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

4

Hi,
Please set bl-output=false and see if it helps:

... ! nvvidconv  bl-output=false ! video/x-raw(memory:NVMM),format=NV12,width=$IMAGEW,height=$IMAGEH,framerate=(fraction)15/1 ! videorate max-rate=15/1 ! ...

And please apply the fix:

Making sure you're not a bot!
[gstreamer] Segmentation fault after restarting gstreamer pipeline

5

Hello,
Thank you for the response, I have just tried that and it doesn’t seem to has any positive effect. The same segfault still.

6

Btw, I have just noticed that my videorate doesn’t have bl-output prop

7

Hi,
On AGX Orin developer kit + dual IMX274 with Jetpack 6.2.1, we run the command and don’t hit segment fault:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
 "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12" ! \
 comp.sink_0 \
 nvarguscamerasrc sensor-id=1 ! \
 "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12" ! \
 comp.sink_1 nvcompositor name=comp background=0 \
 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1200 sink_0::sizing-policy=1 \
 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1200 sink_1::sizing-policy=1 ! \
 "video/x-raw(memory:NVMM),width=3840,height=1200" ! \
 queue ! nvvidconv ! \
 "video/x-raw(memory:NVMM),width=3840,height=1200,format=NV12" ! tee name=timg0 \
 timg0. ! \
 nvvidconv ! "video/x-raw(memory:NVMM),format=NV12,width=3840,height=1200,framerate=(fraction)30/1" ! \
queue ! nvjpegenc ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

Please try your gstreamer pipeline in gst-launch-1.0 and see if it works.

(IMX274 supports 1920x1080 so we configure nvarguscamerasrc to the resolution)

8

Hello,
Thank you for your reply. After a few experiments I must confess it was my bad, I forgot to mention videorate plugin in the first post. Without it everything works fine. However when I try to run your pipeline with videorate added I got the same segfault.

export GST_DEBUG="*:3" 
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
 "video/x-raw(memory:NVMM),width=1920,height=1200,framerate=30/1,format=NV12" ! \
 comp.sink_0 \
 nvarguscamerasrc sensor-id=1 ! \
 "video/x-raw(memory:NVMM),width=1920,height=1200,framerate=30/1,format=NV12" ! \
 comp.sink_1 nvcompositor name=comp background=0 \
 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1200 sink_0::sizing-policy=1 \
 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1200 sink_1::sizing-policy=1 ! \
 "video/x-raw(memory:NVMM),width=3840,height=1200" ! \
 queue ! nvvidconv ! \
 "video/x-raw(memory:NVMM),width=3840,height=1200,format=NV12,framerate=30/1" ! videorate max-rate=15 ! tee name=timg0 \
 timg0. ! \
 nvvidconv ! "video/x-raw(memory:NVMM),format=NV12,width=3840,height=1200,framerate=(fraction)15/1" ! \
 queue ! nvjpegenc ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

From my experiments, what really maters are:

  • Resolution - with 1920x1200 for the last nvvidconv the bug disappears.
  • bl-output=false fixes the problem but requires more compute as far as I found on the forum.
  • videorate and nvvidconv order doesn’t matter.
9

Hi,

It is correct for H264/H265 encoder, but not correct for hardware JPEG encoder. If you send NV12 or I420 in block linear to hardware JPEG encoder, it actually converts to pitch linear through VIC engine in prebuilt lib.

So the issue occurs when videorate max-rate=15 is present. Because the source is in 30fps, does it drop frames to 15fps? Or it re-mux 30fps source into 15fps?

Furthermore you can share source code of nvvidconv and nvjpegenc plugins to check why segment fault occurs. The source code is in

Jetson Linux Release 36.4.4 | NVIDIA Developer
Driver Package (BSP) Sources

10

Hello,
Seems like videorate drop-only flag doesn’t have any effect to the bug. However I set it up to true so frames are only dropped with no duplicates.

Yep, I have already explored nvvidconv and nvjpegenc sources but it mostly seems that the problem is hidden deeper in a proprietary libnvjpeg. However if you have any suggestions I am ready to try.

11

Hi,
If setting bl-output=false works, we would suggest set the property. Hardware JPEG encoder encodes NVMM buffer in pitch linear.