Streaming video over network with opencv and h264 encoder

i am trying to stream live video feed from a camera connected to the Jetson NX to a computer connected to the same network, the network works as wireless ethernet, meaning the jetson sees it as wired connection but in reality its wireless and is limited by bitrate.

On the jetson side, my video write:

cv::VideoWriter video_write_camera_one("appsrc ! video/x-raw,format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx  ! nvvidconv\
 ! video/x-raw(memory:NVMM),format=NV12,width=640,height=360,framerate=30/1 ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=30 \
 bitrate=1800000 EnableTwopassCBR=1 ! h264parse ! rtph264pay ! udpsink host=169.254.84.2 port=5004 auto-multicast=0",\
 cv::CAP_GSTREAMER,30,cv::Size(640,360));

on the receiving computer my video capture is :

cv::VideoCapture video("udpsrc port=5004 auto_multicast=0 !
 application/x-rtp,media=video,encoding-name=H264 ! rtpjitterbuffer latency=0 !
 rtph264depay ! decodebin ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1 "
, cv::CAP_GSTREAMER);

problem is, if the camera shakes a lot or moves a full FOV to any direction, the video stream either freezes or completely pixelates. I was hoping I could get suggestions for either a better encoder(im not limited to nvv4l2h264enc or h264 in general) or a solution for the pixaltion and freezes.

Hi,
Please try small idrinterval. May refer to this post:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

delay isn’t the issue i am facing, although it may be an issue if the delay is too big, i am seeing about 100ms delay which is not the worst. Regardless, the answer you gave to him is fairly similar to my client and server, using the same encode method and similar decode method, which i’ve already tried, and it didn’t really change anything.
The problem seems to be mostly related to how opencv does the compression because i did change the codec in VideoWriter from CAP_GSTREAMER to basic MJPG codec using fourcc and it made it a little bit better, the video tends to freeze less often and the frame only pixelates if the jittering of the camera is bad.
if there is no better encoder than nvv4l2h264enc then i can try to play around with different codecs, but i am hoping for a clearer solution if there is one

Hi,
Please try CBR + setting virtual buffer size:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL

See if the bitrate is more smooth in rapid scene change under the setting.

CBR mode is already on, and vbv-size made things worse honestly.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.