We do not observe long latency in streaming with nvcamerasrc:
Server
$ ./test-launch "nvcamerasrc ! video/x-raw(memory:NVMM),width=640,height=480 ! tee name=t t. ! queue ! nvoverlaysink overlay-x=50 overlay-y=50 overlay-w=640 overlay-h=480 t. ! queue ! omxh264enc ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96"
Client
$ export RTSP_PATH=rtsp://127.0.0.1:8554/test
$ gst-launch-1.0 rtspsrc location="$RTSP_PATH" latency=500 ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 overlay=2
By setting latency=500, we see 500ms delay as expected.
External Media