Hi,
Please set bitrate tp 2Mbps or 4Mbps for a try. Probably the bottleneck is in network bandwidth.
And please try to configure CBR + virtual buffer size:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL
This can avoid burst bitrate in fast scene change.