CSI camera input to opencv and browser simoultaneously

Hi Team,
I have a usecase where I want to process frames into opencv and simoultaneouly display the frame into a flask web app i.e. web browser using a on board CSI camera(raspberry pi camera v2) on jetson nano. I am trying to do it using gstreamer but not able to accomplish the end goal.
The blockers are :

  1. How to display stream from csi camera to browser ? I looked into some ways using vlc plugin but i am not able to build VLC from source . The make commands ends in an error after fixing a lot of errors i.e.

codec/jpeg.c:667:5: error: implicit declaration of function ‘jpeg_mem_dest’; did you mean ‘jpeg_stdio_dest’? [-Werror=implicit-function-declaration]
jpeg_mem_dest(&p_sys->p_jpeg, &p_block->p_buffer, &size);

Anybody has successfully installed VLC on jetson nano ?
The other ways I came across is using WebRTC which require to implement all components of it. I did not deeply explore it but if it is the only way and it works i will definitely look into it.

  1. How to simoultaneously read frames in opencv and web browser ?

I tried a couple of UDP source pipelines and am able to create multiple xvimagesink’s but not sure how to do it in browser and opencv together.

The UDP server command:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=0 ! ‘video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1’ ! omxh264enc control-rate=2 bitrate=4000000 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! h264parse ! rtph264pay mtu=1400 ! udpsink host=0.0.0.0 port=5000 sync=false async=false

The receiver for this server:

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! tee name=t t. ! queue ! xvimagesink t. ! queue ! xvimagesink

Final question is that is this usecase achievable on jetson nano with raspi camera v2 ? If yes, then any guidance would be helpful !
Thanks in advance .

Hi,
There is a sample of showing video preview:

Your usecase is advancing and we don’t have experience on it. We would suggest go to OpenCV forum to get further help.

Thanks for the reply but appsink will not work for our usecase. We did it a workaround using multiple v4l2sinks and creating virtual camera using v4l2loopback.

Hi,
Good to know you have a solution and thanks for sharing it.