I have a usecase where I want to process frames into opencv and simoultaneouly display the frame into a flask web app i.e. web browser using a on board CSI camera(raspberry pi camera v2) on jetson nano. I am trying to do it using gstreamer but not able to accomplish the end goal.
The blockers are :
- How to display stream from csi camera to browser ? I looked into some ways using vlc plugin but i am not able to build VLC from source . The make commands ends in an error after fixing a lot of errors i.e.
codec/jpeg.c:667:5: error: implicit declaration of function ‘jpeg_mem_dest’; did you mean ‘jpeg_stdio_dest’? [-Werror=implicit-function-declaration]
jpeg_mem_dest(&p_sys->p_jpeg, &p_block->p_buffer, &size);
Anybody has successfully installed VLC on jetson nano ?
The other ways I came across is using WebRTC which require to implement all components of it. I did not deeply explore it but if it is the only way and it works i will definitely look into it.
- How to simoultaneously read frames in opencv and web browser ?
I tried a couple of UDP source pipelines and am able to create multiple xvimagesink’s but not sure how to do it in browser and opencv together.
The UDP server command:
gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=0 ! ‘video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1’ ! omxh264enc control-rate=2 bitrate=4000000 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! h264parse ! rtph264pay mtu=1400 ! udpsink host=0.0.0.0 port=5000 sync=false async=false
The receiver for this server:
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! tee name=t t. ! queue ! xvimagesink t. ! queue ! xvimagesink
Final question is that is this usecase achievable on jetson nano with raspi camera v2 ? If yes, then any guidance would be helpful !
Thanks in advance .