nvoverlaysink rendering

Hello,

after considering DaneLLL´s hints

https://devtalk.nvidia.com/default/topic/1046707/jetson-tx2/using-nvoverlaysink-to-overlay-two-input-streams/post/5311509/#5311509

I was able to overlay two camera streams using nvoverlaysink. Yet, a few questions remain.

1. Flash r28.2 on tx2
2. Set win_mask:
    # sudo -s 
    # cd /sys/class/graphics/fb0 
    # echo 4 > blank 
    # echo 0x0 > device/win_mask 
    # echo 0x3f > device/win_mask 
    # echo 0 > blank 
3. Stop x11:
    $ sudo service lightdm stop
4. Run pipeline:
    $ gst-launch-1.0 videotestsrc num-buffers=300 is-live=true ! nvoverlaysink overlay=1 overlay-x=100 overlay-y=100 overlay-w=320 overlay-h=240 & gst-launch-1.0 videotestsrc num-buffers=300 is-live=true ! nvoverlaysink overlay=2 overlay-x=600 overlay-y=600 overlay-w=320 overlay-h=240

Before runnign the pipeline, the x-server gets disabled.

  1. Is nvoverlaysink using the Direct Rendering Manager to draw the scenes? If not, what is it using?
  2. Why is the number of overlays limited to 3 (Hardware Restrictions)?
  3. I wasn´t able to make the overlay on top transparent to enbable blending. How can this be achieved?

Thank you.

Hi,

It controls display controller directly. It is lower layer than DRM.

The limitation is 6. You can run 6 overlays at max.

PLease check if nvcompositor can achieve your usecase.
https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-28-2-ga

Hi,

thanks for the infos. I was looking into the nvcompositor but a few more things are not clear to me.

  1. Transparency still doesn´t work. At least with what I tried. I was using nvcamerasrc and an overlay created with a videotestsrc where I manipulated the alpha channel before compositing. If I turn it to 0, I observe a black rectangle instead of a fully transparent image similar to what I used to observe if using multiple overlays with nvoverlaysink:
gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvvidconv ! nvoverlaysink nvcamerasrc fpsRange="60.0 60.0" sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! nvvidconv ! omxh264enc ! h264parse ! omxh264dec ! comp. videotestsrc ! 'video/x-raw, width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! alpha alpha=0.0  ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! nvvidconv ! omxh264enc ! h264parse ! omxh264dec ! comp.
  1. The avove pipeline doesn´t run smoothly. It seems like the camera freezes immediatley after a few frames/the first frame. The overlay keeps running though. How to make it run smooth?

  2. How can I use the compositor without encode decode? Running

gst-inspect-1.0 nvcompositor

tells me that it supports the NVMM memory in RGBA format. Yet, the following doesn´t work ?? Encoding and Decoding is introducing additional latency we cannot allow plus it occupies the hardware encoder/decoder which we need for other encoding/decoding tasks.

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvvidconv ! nvoverlaysink nvcamerasrc fpsRange="60.0 60.0" sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! comp. nvcamerasrc fpsRange="60.0 60.0" sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! comp.

It is very important for our usecase to be able to create transparent overlays!! If you could help finding a solution it would be greatly appreciated!

Thank you.

Hi,
Alpha blending is not supported in nvcompositor. Please contact us and make feature request.
https://devtalk.nvidia.com/default/topic/1048199/jetson-tx2/gstreamer-glmemory-gt-nvmm-conversion/post/5321339/#5321339

Alpha blending is not working in nvcompositor. Is it my mistake or is it still not supported?

(I’m using the jetson nano, latest SD version jetson-nano-sd-r32.2.1.zip.)

Thank you,
best regards,
Wilhelm

Hi WiSi-Testpilot,

It adds support on r32.2.1, please running below pipeline:

$ gst-launch-1.0 videotestsrc ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0  videotestsrc pattern=1 ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1  nvcompositor name=comp sink_0::alpha=0.5 sink_1::alpha=0.5 ! nvoverlaysink

Hi carolyuu,
thank you very much. It works!
I have included your syntax/suggestions in my script, however I had to add “alpha alpha=1” to each channel.

I have a further question regarding nvarguscamerasrc.
Commands like color-effect=1, saturation, contrast… are unknown.

Please can you give me an example with the correct syntax?
Thank you,
Best regards,
Wilhelm

Hi,
For clearness, you may make a new post for the issue about nvarguscamerasrc

Thank you, I made a new thread:
https://devtalk.nvidia.com/default/topic/1064804/jetson-nano/problem-with-parameters-for-nvarguscamerasrc/