Frames skipping when using nveglstreamsrc and nvvidconv (GPU to CPU)

[ivan.gonzalez1]

Hi,

I’m using the Argus lib to dynamically control the CSI cameras connected to my Nano, and then pass the camera frames to GStreamer via EGLStreams to output the frames to a virtual video device created with v4l2loopback. Although I can see video in the virtual devices, a few frames are dropped every three or four seconds, which is very noticeable when cameras are capturing movement.

After v4l2loopback is installed, I create the virtual video device with this command:

sudo modprobe v4l2loopback video_nr="10" card_label="virtcam" exclusive_caps=1 max_buffers=2

My GStreamer pipeline is the following (see vi_out_v4l2sink.cpp for details):

nveglstreamsrc display=<egl_disp> eglstream=<in_stream> ! nvvidconv ! identity drop-allocation=true ! v4l2sink device=</dev/videoX>

If I replace nveglstreamsrc with nvarguscamerasrc and I run the pipeline with gst-launch-1.0, the bug disappears and the video is fluid, but I cannot control the cameras streams from here.

gst-launch-1.0  nvarguscamerasrc ! nvvidconv ! identity drop-allocation=true ! v4l2sink device=</dev/videoX>

I reduced the frame rate and resolution of the camera frames using Argus to 10fps and 480p, but the bug is still there, so perhaps this is not a performance issue.

In my code ( vi_out_v4l2sink.cpp ), I also noticed that If I change v4l2sink to nvoverlaysink, the bug disappears:

nveglstreamsrc display=<egl_disp> eglstream=<in_stream> ! nvvidconv ! nvoverlaysink

But reappears if I use ximagesink instead:

nveglstreamsrc display=<egl_disp> eglstream=<in_stream> ! nvvidconv ! ximagesink

So maybe it’s something related to nvvidconv moving the buffers produced in nveglstreamsrc from GPU and CPU for v4l2sink to consume.

Could you please help me with this bug? Also, if there are some other alternatives to connect the EGLStream from Argus to the V4L2 virtual device, I’ll be more than happy to try other solutions.

Thanks in advance.
vi_out_v4l2sink.cpp (9.2 KB)

[DaneLLL]

Hi,
We would suggest use nvarguscamerasrc instead of nveglstreamsrc. The source code of nvarguscamerasrc is public. you can add manual controls to it and rebuilt the plugin.

Please download the source code from
https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/sources/t210/public_sources.tbz2

[ivan.gonzalez1]

Thank you @DaneLLL . However, is there something else I could try to make nveglstreamsrc work for my use case?

I want to add a CUDA module in between the Argus module where I control cameras and the GStreamer module with v4l2sink, with EGL streams to transfer the frames from one module to another, so using nvarguscamerasrc wouldn’t be an option for what I want to implement in the future.

Other ideas are very well appreciated.

[ivan.gonzalez1]

Also, I’m reading the source code of nvarguscamerasrc. Perhaps I can find something there that my current Argus module is missing in the configuration of the EGL stream or something related. If I find something, I’ll post it here.
Thanks again @DaneLLL.

[DaneLLL]

Hi,
The nveglstresmsrc is verified in the sample doing video encoding:

/usr/src/jetson_multimedia_api/argus/samples/gstVideoEncode/

Other use-cases are not tested and may not work properly. So we would suggest use nvarguscamerasrc if it can be applied to your use-case. Another advantage is the open source. You can check the source code and do customization through Argus/NvBuffer APIs.

[ivan.gonzalez1]

Thank you @DaneLLL, I’ll replace my current Argus module with a customization of the nvarguscamerasrc as you suggest.