Nvcompositor is not working on Jetpack 4.6

After updating to Jetpack 4.6, nvcompositor is not working.
Two USB cameras are connected to Jetson NX, the gstreamer pipeline is as below:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=I420” ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=I420” ! comp.

The pipeline can not enter into PLAYING state with the error: “nvbuffer_composite Failed”

Enable the GST debug log,I found below error:
0:00:01.512158428 15648 0x557d1eded0 ERROR nvcompositor gstnvcompositor.c:1321:do_nvcomposite: NvBufferComposite failed
0:00:01.512177629 15648 0x557d1eded0 ERROR nvcompositor gstnvcompositor.c:1378:gst_nvcompositor_aggregate_frames: Failed to composit frames

But the same pipeline was working on Jetpack 4.5.

Hi,
We try 2 videotestsrc and 1 videotestsrc + 1 USB camera, and do not hit the error. Will set up 2 USB camras and try.

Hi DaneLLL,

Thanks for your quick feedback.

I have tried 1 videotestsrc + 1 USB camera, and have not hit the issue as well.
But hit the issue for 2 USB cameras and 1 USB camera + 1 CSI camera (nvarguscamerasrc) use cases.

Hi,
Please try

  1. Add queue plugin
  2. Send RGBA buffers to nvcompositor
$ gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),format=NV12,width=640,height=480,framerate=30/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=RGBA" ! queue ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! "video/x-raw(memory:NVMM),format=RGBA" ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=30/1 ! nvvidconv ! "video/x-raw(memory:NVMM),format=RGBA" ! queue ! comp.

Hi DaneLLL,

The point 2 fixes the issue, we converted format to RGBA for the sink pad of nvcompositor, the video streams composited successfully. Thanks a lot!

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=RGBA” ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=RGBA” ! comp.

It solves the issue but it seems to be a workaround, we need to always convert the format to RGBA for the input of nvcompositor, even for format I420 and NV12 which nvcompositor is expected to support on sink pad. Will Nvidia fix it in the later version?

.

1 Like

Hi,
We have added a new property in nvcompositor on Jetpack 4.6:

      interpolation-method: Set interpolation methods
                            flags: readable, writable, controllable
                            Enum "GstInterpolationMethods" Default: 0, "Nearest"
                               (0): Nearest          - Nearest
                               (1): Bilinear         - Bilinear
                               (2): 5-Tap            - 5-Tap
                               (3): 10-Tap           - 10-Tap
                               (4): Smart            - Smart
                               (5): Nicest           - Nicest

If you don’t need it and would like to run the one on Jetpack 4,5, please download the source code of Jetpack 4.5 and rebuild/replace to Jetpack 4.6.

Hi DaneLLL,

Will adding nvvidconv for the format RGBA in between the camerasrc plugin and nvcompositor plugin have any side-effect on performance?

Hi,
The resolution looks to be 640x480. In his resoltuion, the loading should be tiny and hardware converter can handle it to achieve target performance. If you use higher resolution such as 3840x2160 and hit performance issue, please refer to this post:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL
To run hardware converter at max clock always.

Hi DaneLLL,

Do you mean nvvidconv has the same performance issue for 3840x2160 as the DS nvvideoconvert plugin does?

Hi,

This is possible. We enable dynamic frequency scaling for VIC engine and you will see low power consumption in low resolution. But in high-resolution or multi-instance use-case, it is possible to hit the issue. So we suggest run VIC at max clock in the use-case.