I used the MMAPI to write images into all but one of the overlay planes and GStreamer to display a video stream into the remaining plane using nvoverlaysink. The nvoverlaysink output always appears over all the other planes.
My MMAPI test program is derived from tegra_multimedia_api/samples/08_video_dec_drm/video_dec_drm.cpp.
NvDrmFB ui_fb;
uint32_t plane_count = ctx->drm_renderer->getPlaneCount();
// Render a static JPEG image on the first plane
ctx->drm_renderer->createDumbFB(image_w, image_h,
DRM_FORMAT_ARGB8888,
&ui_fb);
for (uint32_t y = 0; y < image_h; ++y)
{
for (uint32_t x = 0; x < image_w; ++x)
{
uint32_t off = ui_fb.bo[0].pitch * y + x * 4;
ui_fb.bo[0].data[off] = *p++;
ui_fb.bo[0].data[off + 1] = *p++;
ui_fb.bo[0].data[off + 2] = *p++;
ui_fb.bo[0].data[off + 3] = *p++;
}
}
for(int i=0;i<plane_count;i++)
if(i != 2)
ctx->drm_renderer->setPlane(i, ui_fb.fb_id,
i * 200, i * 200, image_w, image_h,
0, 0, image_w << 16, image_h << 16);
The above code renders the JPEG as expected with each image overlaying the other, with transparency applied.
The GStreamer pipeline is
gst-launch-1.0 udpsrc port=7891! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec ! nvoverlaysink overlay=3 overlay-x=10 overlay-y=10 overlay-w=800 overlay-h=800
The overlay has to be either 0 or 3.
All other values fail, as expected: NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552.
The overlay configuration code is
cd /sys/class/graphics/fb0
echo 4 > blank
echo 0x0 > device/win_mask
echo 0x3f > device/win_mask
echo 0 > blank
systemctl stop lightdm
The DRM overlay planes correctly overlay each other.
The nvoverlaysinks correctly overlay each other, w/o transparency.
The combination does not work as expected.
As far as I can tell, the documentation does not describe this interaction.