How to Optimize omxh264enc to reduce RTSP latency

I’m processing an RTSP stream from a wireless IP Camera processing (or plan to ) the stream with OpenCV and pushing the processed streamed to an RTSP server with Gstreamer. Although I’m using the AGX hardware encorder (omxh264enc), I still have about 7 seconds of latency.

Is there anything I can do to optimize the omxh264enc setting to reduce the latency?

Can you make general recommendations on how I can reduce the pipeline’s latency?

My setup:
AGX, OpenCV with Gstreamer.

My Code:

import numpy as np
from multiprocessing import Process
import cv2

cap = cv2.VideoCapture(‘rtsp://192.168.0.6:8080/h264_pcm.sdp’)

gst_str= ("appsrc ! autovideoconvert ! omxh264enc bitrate=4000000 ! video/x-h264 ! rtspclientsink location=rtsp://localhost:8554/mystream ");

four_cc = cv2.VideoWriter_fourcc(*‘X264’)
out = cv2.VideoWriter(gst_str, cv2.CAP_GSTREAMER,four_cc, 30, (1280, 720))

while(cap.isOpened()):
ret, frame = cap.read()
out.write(frame)
if cv2.waitKey(20) & 0xFF == ord(‘q’):
break
cap.release()
out.release()
cv2.destroyAllWindows()

Thank you,

What RTSP server and player are you using? I.e. If you are using VLC or ffplay, you have to reduce size of buffer.

I’m using VLC and have reduced the cache setting to zero but still experiencing 7 seconds of latency. Any other suggestions.

Thanks!

I’m using opensource rtsp-simple-server RTSP server from github:

Hi,
Please try UDP streaming. Here is a discussion about latency:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

And looks like there is buffering mechanism in RTSP and there is a latency. We can see very low latency in UDP. Please give it a try.

Try
ffplay -fflags nobuffer rtsp://….
Even vlc set with zero cache , there is still a considerable amount of latency