Unable to overlay text when using udpsrc

This command displays the text overlaid on test video source.

gst-launch-1.0 videotestsrc  ! textoverlay text ="HELLO" ! nvoverlaysink

This code displays my multicast MPEG TS stream

gst-launch-1.0 udpsrc  port=4321  ! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec  !   nvoverlaysink

Adding the textoverlay as follows does not work

gst-launch-1.0 udpsrc  port=4321  ! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec  ! textoverlay text="Hello" !   nvoverlaysink

The output indicates the pipeline is working but nothing is displayed.

The log outputs are identical

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
TVMR: cbBeginSequence: 1179: BeginSequence  1920x1088, bVPR = 0
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1529: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1600: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1669: ColorFormat : 5
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 12, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 1920x1088 (x 12), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088
TVMR: FrameRate = 30
TVMR: NVDEC LowCorner Freq = (180000 * 1024)
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 30.077088
TVMR: FrameRate = 30.000930
TVMR: FrameRate = 29.998860
TVMR: FrameRate = 29.997960
handling interrupt.
Interrupt: Stopping pipeline ...

Jetson TX2
Linux tegra-ubuntu 4.4.38-tegra #1 SMP PREEMPT Thu May 17 00:15:19 PDT 2018 aarch64 aarch64 aarch64 GNU/Linux
GStreamer 1.8.3

Probably textoverlay uses CPU memory, but nvoverlaysink may expect NVMM (contiguous) memory only for some formats.
Maybe adding nvvidconv between these could improve. It will copy from CPU memory to NVMM memory.

You may also use -v with gst-launch so that you know which format is used for each plugin, and you may also try to specify caps for being sure it does the way you think it should.

With -v

gst-launch-1.0 -v udpsrc  port=4321  ! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec  ! textoverlay text="Hello" !   nvoverlaysink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)30/1\,\ parsed\=\(boolean\)true\,\ profile\=\(string\)high\,\ level\=\(string\)4.1"
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)30/1\,\ parsed\=\(boolean\)true\,\ profile\=\(string\)high\,\ level\=\(string\)4.1"
TVMR: cbBeginSequence: 1179: BeginSequence  1920x1088, bVPR = 0
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1529: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1600: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1669: ColorFormat : 5
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 12, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 1920x1088 (x 12), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstTextOverlay:textoverlay0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstTextOverlay:textoverlay0.GstPad:video_sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
TVMR: FrameRate = 30
TVMR: NVDEC LowCorner Freq = (180000 * 1024)
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 30.062982
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:05.752910089
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done
Setting pipeline to NULL ...
Freeing pipeline ...

Try:

gst-launch-1.0 -v udpsrc  port=4321  ! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec  ! textoverlay text="Hello" ! <b>nvvidconv</b> ! nvoverlaysink

GStreamer cannot handle interlaced stream, which triggers all the asserts
I used the wrong test source, sorry.

Testing with 1080p stream

gst-launch-1.0 -v udpsrc  port=4321  ! tsdemux name=demux demux. ! queue max-size-time=0 max-size-bytes=0 max-size-buffers=0 ! h264parse ! omxh264dec  ! textoverlay text="Hello" ! nvvidconv ! nvoverlaysink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)nal"
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)30/1\,\ parsed\=\(boolean\)true\,\ profile\=\(string\)high\,\ level\=\(string\)4.1"
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)30/1\,\ parsed\=\(boolean\)true\,\ profile\=\(string\)high\,\ level\=\(string\)4.1"
TVMR: cbBeginSequence: 1179: BeginSequence  1920x1088, bVPR = 0
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1529: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1600: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1669: ColorFormat : 5
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 12, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 1920x1088 (x 12), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_mini_object_copy: assertion 'mini_object != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_structure_copy: assertion 'structure != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_append_structure_full: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_structure_copy: assertion 'structure != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_append_structure_full: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_intersect_full: assertion 'GST_IS_CAPS (caps1)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_mini_object_copy: assertion 'mini_object != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_structure_copy: assertion 'structure != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_append_structure_full: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_structure_copy: assertion 'structure != NULL' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_append_structure_full: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_caps_intersect_full: assertion 'GST_IS_CAPS (caps1)' failed

(gst-launch-1.0:5258): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstTextOverlay:textoverlay0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstTextOverlay:textoverlay0.GstPad:video_sink: caps = "video/x-raw\,\ format\=\(string\)NV12\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)30/1"
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-omx/omx/gstomxvideodec.c(2904): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-supported
Execution ended after 0:00:00.613009740
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done
Setting pipeline to NULL ...
Freeing pipeline ...

Not tried on a TX2, but on Xavier with R31.1 (GStreamer 1.14.1) this works:

#sender
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM)' ! omxh264enc ! h264parse ! rtph264pay config-interval=1 ! udpsink host=127.0.0.1 port=5000
#receiver
gst-launch-1.0 -v udpsrc port=5000 ! 'application/x-rtp, encoding-name=H264, payload=96' ! rtph264depay ! h264parse ! omxh264dec ! textoverlay text ="HELLO" ! nvvidconv ! nvoverlaysink

Does this work on your TX2 ? Are you running L4T R28.2.1 ?

[EDIT: Tested this on a TX2 R28.2.1:

#sender
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=640, height=480' ! omxh264enc ! h264parse ! rtph264pay config-interval=1 ! udpsink host=127.0.0.1 port=5000
#receiver
gst-launch-1.0 -v udpsrc port=5000 ! 'application/x-rtp, encoding-name=H264, payload=96' ! rtph264depay ! h264parse ! omxh264dec ! 'video/x-raw(memory:NVMM), format=NV12' ! nvvidconv ! 'video/x-raw' ! textoverlay text ="HELLO" ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvoverlaysink

]

I am running L4T R28.2.1.

The TX2 specific commands work fine.
I really appreciate the assistance, thank you.

I now have the desired overlay text over my MPEG TS multicast stream.
The CPU load jumped from 15% to 79%, which is not that surprising.
Unfortunately, that is too high for my application.

I will need to find another mechanism to perform the overlay.

You can boost your Jetson if not yet done with:

sudo nvpmodel -m 0
sudo /home/nvidia/jetson_clocks.sh

but it may draw much more power.

You may also try to use nvoverlaysink overlays. If you look at this plugin options with gst-inspect-1.0 you should see an option overlay with two possible values 1 & 2. You may use one pipeline for decoding your ts stream and display on overlay 1 and use a second pipeline for text that would use overlay 2:

gst-launch-1.0 -v udpsrc port=5000 ! 'application/x-rtp, encoding-name=H264, payload=96' ! rtph264depay ! h264parse ! omxh264dec ! 'video/x-raw(memory:NVMM), format=NV12' ! nvoverlaysink overlay=1        textoverlay text ="HELLO" ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvoverlaysink overlay=2

(Not tested if textoverlay can work without input, I’m currently away from any Jetson)

The only reason we are considering the TX2 is to deliver a compact, very low power system.

Without the overlays, I can achieve sub 7W.

I am already using the overlays to display 4 video streams, one per overlay.

The textoverlay without a video source does not report any errors, but does not display anything.
Adding videotestsrc to provide the background works but the overlay completely blocks the overlay below it.

I am attempting to display a PNG image with transparency in the uppermost overlay, so far without success.

gst-launch-1.0 -v videotestsrc ! nvoverlaysink overlay=1
gst-launch-1.0 -v filesrc location=Transparency_example.PNG ! pngdec  ! imagefreeze  ! nvvidconv  ! nvoverlaysink overlay=4

Where the png is sourced from https://upload.wikimedia.org/wikipedia/commons/6/66/Transparency_example.PNG

I have also tried alpha

The dump from the second command looks reasonable

gst-launch-1.0 -v filesrc location=Transparency_example.PNG ! pngdec  ! imagefreeze  ! nvvidconv  ! nvoverlaysink overlay=4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstPngDec:pngdec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)RGBA\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)sRGB\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:src: caps = "video/x-raw\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)RGBA\,\ colorimetry\=\(string\)sRGB"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)RGBA"
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)RGBA"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = "video/x-raw\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)25/1\,\ format\=\(string\)RGBA\,\ colorimetry\=\(string\)sRGB"
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)RGBA\,\ width\=\(int\)194\,\ height\=\(int\)68\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)sRGB\,\ framerate\=\(fraction\)0/1"
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Hi,
textoverlay processes on CPU buffers(video/x-raw), and triggers certain CPU load.

You may try to use nvivafilter to implement the drawing on NVMM buffers for better performance.
https://devtalk.nvidia.com/default/topic/963123/jetson-tx1/video-mapping-on-jetson-tx1/post/4979740/#4979740
https://devtalk.nvidia.com/default/topic/978438/jetson-tx1/optimizing-access-to-image-data-acquired-with-nvcamerasrc/post/5026998/#5026998

That seems rather complex and will probably increase the TDP.

I have two unused overlays in the framebuffer.

My next approach is to try drawing the required text in one of those overlays.
That would provide the desired textoverlay w/o any additional GPU or CPU load.
Can you point me to examples and/or documentation for accessing the TX2 framebuffer?

Alternatively, I can use GStreamer to generate the desired overlay pane but that requires that the pipeline honors the alpha component in the source image. See #10
So far I have been unable to get that working.

Hi,
Attach a sample code of using CUDA to apply alpha effect.
You can replace it, rebuild libnvsample_cudaprocess.so, and run:

$ gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! "video/x-raw(memory:NVMM), format=(string)RGBA" ! nvoverlaysink

For other suggestions, you may go to gstreamer forum http://gstreamer-devel.966125.n4.nabble.com/
There could be other elements supporting this functionality.

nvsample_cudaprocess.zip (2.75 KB)

There is a small typo: RGBA should be NV12

Power consumption increases by 33%, but still falls in acceptable range.

It may give different format, paths and results if using EGL output (RGBA)

gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! "video/x-raw(memory:NVMM), format=(string)RGBA" ! nvegltransform ! nveglglessink

or NV12

gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! "video/x-raw(memory:NVMM), format=(string)NV12" ! nvoverlaysink