Tx2-4g R32.3.1 Need "fast textoverlay" in video creation stream

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

For my application I need to do “fast textoverlay” functionality to NVMM,format=nv12 buffers that are being passed to h264enc, It seems that most filters depend on the gpu/display and do not match my app requirements.

Any suggestions on what to look at for basically modifying NVMM buffers in a gstreamer filter? Glanced at multimedia api, nothing jumped off the page,

Guess I am asking for a peak into the Nvidia low level stuff. textoverlay, clockoverlay, nvivafilter(Nvidia), emboverlay, qtoverlay(Ridgerun) are slow in the video stream and aimed at using gpu/display where they work correctly.

2

Hi @terrysu50z,

May I know what are your app requirements?

You can check RidgeRun’s Emboverlay which is fast gstreamer plugin for add text or other stuff

You can also check RidgeRun’s QtOverlay, we recently added NVMM support for it.

Regards,
Fabian
www.ridgerun.com

3

Fabian,

Thanks for the view and pointer.

I tried to mention the Ridgerun overlays in the original post, both emboverlay and qtoverlay work great if you are modifying frames that will be displayed. They both perform similar to textoverlay and clockoverlay in the camera to h264enc path.

I actually purchased emboverlay, found it did not work, and am currently working with Leon to get qtoverlay to work, Leon is a god sent in the support area, but things are looking like my requirements have not been addressed yet in the nvidia gstreamer filter area.

Leon from ridgerun is GREAT!.

This is a new product possibility for Ridgerun :->

Terry

1 Like
4

Hi Terry,

Thanks for the kind words on Leon support. Actually QtOverlay was created to solve the problem of creating overlays for frames that are NOT displayed but encoded. We are working in two example pipelines to show you how does it work. Other existing customers are already using it for similar requirements.

-David

5

Hi @terrysu50z

Thanks for your kind words. I just wanted to notify that I have provided an example that can help you in your use case.

Basically, the reason why emboverlay was not suitable for your use case was because of the buffer kind (it does not manage NVMM) and it’s intended for constrained embedded systems. Jetson are actually high-performance embedded systems and GstQtOverlay is the best way to go since it utilises the resources much better.

About the display reason, and complementing David’s answer, I just wanted to let you know about this case:

gst-launch-1.0 videotestsrc \
  ! nvvidconv \
  ! 'video/x-raw(memory:NVMM)' \
  ! nvvidconv \
  ! 'video/x-raw(memory:NVMM),format=RGBA, width=1920, height=1080' \
  ! qtoverlay qml=main.qml \
  ! perf \
  ! nvvidconv  \
  ! nvv4l2h264enc  maxperf-enable=1 bitrate=8000000 \
  ! 'video/x-h264, stream-format="byte-stream"' \
  ! h264parse \
  ! qtmux \
  ! filesink location="test.mp4" -e

I have just tested this pipeline on a TX2 and it worked without display intervention and it runs at 100fps. I pointed out some requirements in terms of the environment you have to follow to make it work for your case.

I would be glad to continue on the support line we were following before.

Best regards,
Leon

6

​Leon,

nvidia system crashed and I am having some problems, so I don’t know when I will be able to try the info in this email, Just to let you know I am not ignoring you, but I might be silent for a while.

Sorry, will be back in a while.

Terry