Hi,
RTSP source is compressed video stream and there is latency from buffering/decoding. It is expected to have additional latency when comparing to Bayer camera source. You may try to configure latency to 150 or 300 ms for a try. Please not there may be packet drop if the setting is too small:
Multi stream video artifacts - #3 by Fiona.Chen