nvoverlaysink rendering

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hello,

after considering DaneLLL´s hints

https://devtalk.nvidia.com/default/topic/1046707/jetson-tx2/using-nvoverlaysink-to-overlay-two-input-streams/post/5311509/#5311509

I was able to overlay two camera streams using nvoverlaysink. Yet, a few questions remain.

1. Flash r28.2 on tx2
2. Set win_mask:
    # sudo -s 
    # cd /sys/class/graphics/fb0 
    # echo 4 > blank 
    # echo 0x0 > device/win_mask 
    # echo 0x3f > device/win_mask 
    # echo 0 > blank 
3. Stop x11:
    $ sudo service lightdm stop
4. Run pipeline:
    $ gst-launch-1.0 videotestsrc num-buffers=300 is-live=true ! nvoverlaysink overlay=1 overlay-x=100 overlay-y=100 overlay-w=320 overlay-h=240 & gst-launch-1.0 videotestsrc num-buffers=300 is-live=true ! nvoverlaysink overlay=2 overlay-x=600 overlay-y=600 overlay-w=320 overlay-h=240

Before runnign the pipeline, the x-server gets disabled.

  1. Is nvoverlaysink using the Direct Rendering Manager to draw the scenes? If not, what is it using?
  2. Why is the number of overlays limited to 3 (Hardware Restrictions)?
  3. I wasn´t able to make the overlay on top transparent to enbable blending. How can this be achieved?

Thank you.

2

Hi,

It controls display controller directly. It is lower layer than DRM.

The limitation is 6. You can run 6 overlays at max.

PLease check if nvcompositor can achieve your usecase.

3

Hi,

thanks for the infos. I was looking into the nvcompositor but a few more things are not clear to me.

  1. Transparency still doesn´t work. At least with what I tried. I was using nvcamerasrc and an overlay created with a videotestsrc where I manipulated the alpha channel before compositing. If I turn it to 0, I observe a black rectangle instead of a fully transparent image similar to what I used to observe if using multiple overlays with nvoverlaysink:
gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvvidconv ! nvoverlaysink nvcamerasrc fpsRange="60.0 60.0" sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! nvvidconv ! omxh264enc ! h264parse ! omxh264dec ! comp. videotestsrc ! 'video/x-raw, width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! alpha alpha=0.0  ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! nvvidconv ! omxh264enc ! h264parse ! omxh264dec ! comp.

  1. The avove pipeline doesn´t run smoothly. It seems like the camera freezes immediatley after a few frames/the first frame. The overlay keeps running though. How to make it run smooth?

  2. How can I use the compositor without encode decode? Running

gst-inspect-1.0 nvcompositor

tells me that it supports the NVMM memory in RGBA format. Yet, the following doesn´t work ?? Encoding and Decoding is introducing additional latency we cannot allow plus it occupies the hardware encoder/decoder which we need for other encoding/decoding tasks.

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvvidconv ! nvoverlaysink nvcamerasrc fpsRange="60.0 60.0" sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! comp. nvcamerasrc fpsRange="60.0 60.0" sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)60/1' ! comp.

It is very important for our usecase to be able to create transparent overlays!! If you could help finding a solution it would be greatly appreciated!

Thank you.

4

Hi,
Alpha blending is not supported in nvcompositor. Please contact us and make feature request.
[url]https://devtalk.nvidia.com/default/topic/1048199/jetson-tx2/gstreamer-glmemory-gt-nvmm-conversion/post/5321339/#5321339[/url]

5

Alpha blending is not working in nvcompositor. Is it my mistake or is it still not supported?

(I’m using the jetson nano, latest SD version jetson-nano-sd-r32.2.1.zip.)

Thank you,
best regards,
Wilhelm

6

Hi WiSi-Testpilot,

It adds support on r32.2.1, please running below pipeline:

$ gst-launch-1.0 videotestsrc ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0  videotestsrc pattern=1 ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1  nvcompositor name=comp sink_0::alpha=0.5 sink_1::alpha=0.5 ! nvoverlaysink
1 Like
7

Hi carolyuu,
thank you very much. It works!
I have included your syntax/suggestions in my script, however I had to add “alpha alpha=1” to each channel.

I have a further question regarding nvarguscamerasrc.
Commands like color-effect=1, saturation, contrast… are unknown.

Please can you give me an example with the correct syntax?
Thank you,
Best regards,
Wilhelm

8

Hi,
For clearness, you may make a new post for the issue about nvarguscamerasrc

9

Thank you, I made a new thread: