Nvcompositor is not working on Jetpack 4.6

liuying1 · October 8, 2021, 7:41am

After updating to Jetpack 4.6, nvcompositor is not working.
Two USB cameras are connected to Jetson NX, the gstreamer pipeline is as below:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=I420” ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=I420” ! comp.

The pipeline can not enter into PLAYING state with the error: “nvbuffer_composite Failed”

Enable the GST debug log，I found below error:
0:00:01.512158428 15648 0x557d1eded0 ERROR nvcompositor gstnvcompositor.c:1321:do_nvcomposite: NvBufferComposite failed
0:00:01.512177629 15648 0x557d1eded0 ERROR nvcompositor gstnvcompositor.c:1378:gst_nvcompositor_aggregate_frames: Failed to composit frames

But the same pipeline was working on Jetpack 4.5.

DaneLLL · October 8, 2021, 8:38am

Hi,
We try 2 videotestsrc and 1 videotestsrc + 1 USB camera, and do not hit the error. Will set up 2 USB camras and try.

liuying1 · October 8, 2021, 8:44am

Hi DaneLLL,

Thanks for your quick feedback.

I have tried 1 videotestsrc + 1 USB camera, and have not hit the issue as well.
But hit the issue for 2 USB cameras and 1 USB camera + 1 CSI camera (nvarguscamerasrc) use cases.

DaneLLL · October 8, 2021, 9:10am

Hi,
Please try

Add queue plugin
Send RGBA buffers to nvcompositor

$ gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),format=NV12,width=640,height=480,framerate=30/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=RGBA" ! queue ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! "video/x-raw(memory:NVMM),format=RGBA" ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=30/1 ! nvvidconv ! "video/x-raw(memory:NVMM),format=RGBA" ! queue ! comp.

liuying1 · October 8, 2021, 9:35am

Hi DaneLLL,

The point 2 fixes the issue, we converted format to RGBA for the sink pad of nvcompositor, the video streams composited successfully. Thanks a lot!

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=RGBA” ! nvcompositor name=comp sink_0::width=640 sink_0::height=480 sink_0::xpos=0 sink_0::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::xpos=640 sink_1::ypos=0 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! “video/x-raw(memory:NVMM),format=RGBA” ! comp.

It solves the issue but it seems to be a workaround, we need to always convert the format to RGBA for the input of nvcompositor, even for format I420 and NV12 which nvcompositor is expected to support on sink pad. Will Nvidia fix it in the later version?

.

DaneLLL · October 13, 2021, 2:59am

Hi,
We have added a new property in nvcompositor on Jetpack 4.6:

      interpolation-method: Set interpolation methods
                            flags: readable, writable, controllable
                            Enum "GstInterpolationMethods" Default: 0, "Nearest"
                               (0): Nearest          - Nearest
                               (1): Bilinear         - Bilinear
                               (2): 5-Tap            - 5-Tap
                               (3): 10-Tap           - 10-Tap
                               (4): Smart            - Smart
                               (5): Nicest           - Nicest

If you don’t need it and would like to run the one on Jetpack 4,5, please download the source code of Jetpack 4.5 and rebuild/replace to Jetpack 4.6.

liuying1 · October 13, 2021, 6:03am

Hi DaneLLL，

Will adding nvvidconv for the format RGBA in between the camerasrc plugin and nvcompositor plugin have any side-effect on performance?

DaneLLL · October 13, 2021, 9:13am

Hi,
The resolution looks to be 640x480. In his resoltuion, the loading should be tiny and hardware converter can handle it to achieve target performance. If you use higher resolution such as 3840x2160 and hit performance issue, please refer to this post:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL
To run hardware converter at max clock always.

liuying1 · October 13, 2021, 9:45am

Hi DaneLLL，

Do you mean nvvidconv has the same performance issue for 3840x2160 as the DS nvvideoconvert plugin does?

DaneLLL · October 14, 2021, 2:04am

Hi,

This is possible. We enable dynamic frequency scaling for VIC engine and you will see low power consumption in low resolution. But in high-resolution or multi-instance use-case, it is possible to hit the issue. So we suggest run VIC at max clock in the use-case.

system · November 4, 2021, 5:31am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.