Fullscreen borderless video sink ? Porting nvoverlaysink to Jetpack 5.1.2?

• Jetson Xavier NX DevKit
**• DeepStream Version 6.3
**• JetPack 5.1.2

Hi,

basically i need a fullscreen borderless sink for my use-case. I was using nvoverlaysink in JP4.6.4 and it was and still is perfect for my use-case, it can run from GUI, self adjust the resolution to a capture card and i don’t need to restart gdm if i need to do something.

I know that nvdrmvideosink exists, but it cannot run from GUI and also somehow flickers when launching my code from python bindings, also flickers on plane 1 and plane 2.

Maybe someone has an idea or experience porting nvoverlaysink to jetpack 5.1.2 ? or maybe another alternative ?

I have managed to install the gst-egl and gst-omx1 from JP4.6.4 public_sources.tbz2, hoping it would bring nvoverlaysink back, but the .so files end up getting blacklisted from gstreamer. GST-DEBUG (after deleting cache) is not giving any info to why exactly the plugin is being blacklisted.

Any input is much appreciated, thanks!

Could you try to use the nv3dsink and see if it meets your needs?

Hello,

Sadly not really, i need a direct rendering sink

Thanks!

The nvoverlaysink is deprecated long ago. Could you refer to the link below and adjust the parameters to meet your needs?
https://docs.nvidia.com/jetson/archives/r36.2/DeveloperGuide/SD/Multimedia/AcceleratedGstreamer.html?highlight=nvdrmvideosink#video-playback-examples

nvdrmvideosink supports these properties

conn_id: Set the connector ID for the display.

plane_id: Set the plane ID.

set_mode: Set the default mode (resolution) for playback.

sadly it doesn’t work for my use case.

so there is no way to get a sink resembling nvoverlaysink in Jetpack 5 ?

Yes. It’s deprecated. Could you again describe your requirements in detail and why nvdrmvideosink doesn’t meet your use case? We’ll check that.

basically my program has 3 sources that will be composited:

source 1,2,3 → nvcompositor → textoverlay → textoverlay → sink

the output will be sent to an HDMI capture card to be recorded by a laptop as a virtual camera.

Things i find from about nvoverlaysink:

  1. With my program nvdrmvideosink can output the video, but it flickers. I am not sure why that is
  2. If im not mistaken, nvoverlaysink can self adjust the resolution so that it always gives a fullscreen on the hdmi capture card output. With nvdrmvideosink i have to adjust manually from the python program so that uses exactly the right resolution, not bigger not smaller
  3. this is not a big problem, but it’s practical that nvoverlaysink can also be run on GUI

Can you help to setup the nvdrmvideosink for this case ?

  1. For such program (with nvcompositor & textoverlay), do i need to link some elements before the nvdrmvideosink ?
  2. What does “set-mode” actually do ? i am not too sure from the description and everytime i set “set-mode = 1”, i only get a black screen

Thanks for the support!

You can use the gst-inspect-1.0 nvdrmvideosink to check the set-mode parameter.

  set-mode            : Selects whether user wants to choose the default mode which is
                        already set by connector (set_mode = 0) or wants to select the mode
                        of the video stream (set_mode = 1). In the latter case, error is
                        thrown when the input stream resolution does not match with
                        the supported modes of the connector.

Nvdrmvideosink cannot be run along with another GUI system.

Can you also help me with the flickering i mentioned ?

with this pipeline i get flickering:
source 1,2,3 → nvcompositor → textoverlay → textoverlay → sink

I’ve tried a couple small tests to try and pin-point the problem:

  1. Any source → nvcompositor → nvdrmvideosink no flicker
  2. Source 1, 2, 3 → nvcompositor → nvdrmvideosink no flicker
  3. Source 1, 2, 3 → nvcompositor → CPU memory → xvimagesink no flicker
  4. Source 1, 2, 3 → nvcompositor → CPU memory → textoverlay → NVMM → nvdrmvideosink flicker
  5. Source 1, 2, 3 → nvcompositor → CPU memory → NVMM → nvdrmvideosink flicker

It seems i cannot convert to CPU memory to use nvdrmvideosink, but i must have the textoverlay for my program.

Below i attached a short video (12 seconds) in a .zip to show you what the problem looks like.

test.zip (8.1 MB)

Thanks!

OK. Could you attach the gst-launch-1.0 command you used? I have tried the command below, it works fine.

gst-launch-1.0 nvcompositor name=comp     sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540     sink_1::xpos=960 sink_1::ypos=0 sink_1::width=960 sink_1::height=540     sink_2::xpos=0 sink_2::ypos=540 sink_2::width=960 sink_2::height=540     sink_3::xpos=960 sink_3::ypos=540 sink_3::width=960 sink_3::height=540   ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! 'video/x-raw,format=RGBA' ! textoverlay text="hello" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvdrmvideosink   uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 source::latency=0 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! queue ! comp.sink_0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 source::latency=0 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! queue ! comp.sink_1 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 source::latency=0 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! queue ! comp.sink_2

Pipeline after the compositor with flicker:

nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=300 sink_1::width=800 sink_1::height=600 sink_2::xpos=1120 sink_2::ypos=300 sink_2::width=800 sink_2::height=600 ! nvvideoconvert ! ‘video/x-raw, width=1920, height=1080’ ! textoverlay text=“Test” ! nvvideoconvert ! ‘video/x-raw(memory:NVMM)’ ! nvdrmvideosink conn-id=1 plane-id=1 set-mode=0 sync=True

I also found out in the meantime that this pipeline works without flicker with nvvidconv instead of nvvideoconvert to convert to CPU before textoverlay:

nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=300 sink_1::width=800 sink_1::height=600 sink_2::xpos=1120 sink_2::ypos=300 sink_2::width=800 sink_2::height=600 ! nvvidconv ! ‘video/x-raw, width=1920, height=1080’ ! textoverlay text=“Test” ! nvvideoconvert ! ‘video/x-raw(memory:NVMM)’ ! nvdrmvideosink conn-id=1 plane-id=1 set-mode=0 sync=True

And i think, also looking at the pipeline you gave with nvvidconv, confirmed that nvvideoconvert was the problem. At the moment, all conversion in my pipeline is done by nvvideoconvert (well, now except the one before textoverlay).

What is actually the difference between nvvideoconvert and nvvidconv ? I don’t see any big difference yet in performance, but is there a performance difference expected between nvvidconv and nvvideoconvert ?

Nvvidconv is for jetson AcceleratedGstreamer.
Nvvideoconvert is for DeepStream.
You can refer to the FAQ.

So my pipeline is mixed with Nvidia plugins (nvinfer, nvcompositor, …) and Gstreamer standard plugins (textoverlay, filesrc, …), i think for my case then i would need to look for accelerated Gstreamer instead of Deepstream, if i understand correctly ?

should i use nvvidconv for every conversion ? or nvvideoconvert where it works and just use nvvidconv if nvvideoconvert does not work ?

Thanks!

No.As I attached before FAQ, with these incompatible plugins, you’d better use the accelerated Gstreamer.

Okay, thank you so much!
I was also able to get nvdrmvideosink to self-adjust the resolution with set-mode=1

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.