Hi,
We would suggest try to set CBR + virtual buffer size:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL
This is to balance bitrate and video quality. Please give it a try and see if it can help the use-case. And please use latest Jetpack 5.1.3 or 4.6.4.