Is OpenCV VideoCapture able to get branched appsink from test-launch

[changwen.xie]

Hi
I can use OpenCV VideoCapture to launch the gstreamer script below to carry out the image processing and streaming video to other device with WIFI connection.

"v4l2src device=/dev/video0 \
    ! video/x-raw, format=UYVY, width=2592, height=1944, framerate=24/1 \
    ! nvvidconv ! video/x-raw(memory:NVMM), format=I420 \
    ! tee name=camNVMM ! queue ! nvv4l2h264enc maxperf-enable=true insert-vui=true insert-sps-pps=1 bitrate=10000000 control-rate=1 vbv-size=625000 \
    preset-level = 1  MeasureEncoderLatency=1 \
    ! h264parse ! rtph264pay ! udpsink clients=192.168.18.18:38298 \
    camNVMM. ! queue ! nvvidconv ! video/x-raw, format=GRAY8, width=640,height=480 ! appsink"

The video streaming is using UDP. When tried to stream the video to other device using RTSP,
I tried the code below:

./test-launch "v4l2src device=/dev/video0 \
! video/x-raw, format=(string)UYVY, width=(int)2592, height=(int)1944,framerate=24/1 \
! nvvidconv \
! video/x-raw(memory:NVMM),format=(string)I420 \
! tee name=camNVMM ! queue ! nvv4l2h264enc preset-level=1 MeasureEncoderLatency=1 maxperf-enable=true insert-vui=true insert-sps-pps=1 bitrate=10000000  control-rate=1 vbv-size=625000 speed-preset=ultrafast \
! h264parse ! rtph264pay name=pay0 "

I can get the RTSP video from other WIFI connected device.
The issue is that how can I get the video frame from

camNVMM. ! queue ! nvvidconv ! video/x-raw, format=GRAY8, width=640,height=480 ! appsink

by using OpenCV VideoCaptued) for my image processing?
Thanks

CX

[DaneLLL]

Hi,
Please refer to this python sample:
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL

[changwen.xie]

Thanks. Your suggestion should work, but it added a latency on image processing.
I am thinking another method may also work, i.e. update my OpenCV VideoCapture gstreamer script to stream the video to udpsink at local host, and then using test-launch get the udpsrc from local host and stream to other device. But it will increase the video stream latency.
Is there any other better method can do the job?

[DaneLLL]

For low latency, it may be better to run both server and client in gstreamer commands. As we have verified and suggested in
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

Running with OpenCV may have certain latency. And it requires much CPU usage. May hit performance constraint on Jetson Nano.