Do you have 30 cameras? What is the original FPS of your cameras?
Have you set proper delay with your rtsp client?
i use 10 to 25 cams at a time, mostly have 25fps, some of them 10 and some of them are 30 fps.
But like i said, i convert all to 10 fps before feeding into nvstreammux
Where and how did you convert all videos into 10 FPS?
just before nvstreammux.
How did you change the FPS?
using videorate and capsfilter
def create_source_bin(index,decoder_configs):
# print("Creating source bin")
# Create a source GstBin to abstract this bin's content from the rest of the
# pipeline
bin_name = "source_bin-%02d" % index
FILE_LOGGER.info(f"Creating source bin with name: {bin_name}")
nbin = Gst.Bin.new(bin_name)
if not nbin:
sys.stderr.write(" Unable to create source bin \n")
# Source element for reading from the uri.
# We will use decodebin and let it figure out the container format of the
# stream and the codec and plug the appropriate demux and decode plugins.
uri_decode_bin = Gst.ElementFactory.make("uridecodebin", f"uri_decode_bin-{index}")
if not uri_decode_bin:
sys.stderr.write(" Unable to create uri decode bin \n")
filename = cam_idx_to_uri_map[index]
# We set the input uri to the source element
if "rtsp" in filename:
uri_decode_bin.set_property("uri", filename)
else:
if not filename.startswith("file://"):
filename = f"file:///{config.MP4_LOCATION}/{filename}"
FILE_LOGGER.info(f"new file name for DE-({index}) is {filename}")
uri_decode_bin.set_property("uri", filename)
# Connect to the "pad-added" signal of the decodebin which generates a
# callback once a new pad for raw data has beed created by the decodebin
uri_decode_bin.set_property('force-sw-decoders', 'true')
uri_decode_bin.connect("pad-added", cb_newpad, nbin,index)
uri_decode_bin.connect("child-added", decodebin_child_added,index,decoder_configs)
# Create elements for nvvideoconvert, videorate, and queue
nvvideoconvert = Gst.ElementFactory.make("nvvideoconvert", "nvvideoconvert")
videorate = Gst.ElementFactory.make("videorate", "videorate")
capsfilter = Gst.ElementFactory.make("capsfilter", "capsfilter")
nvvideoconvert2 = Gst.ElementFactory.make("nvvideoconvert", "nvvideoconvert2")
capsfilter2 = Gst.ElementFactory.make("capsfilter", "capsfilter2")
queue = Gst.ElementFactory.make("queue", "queue")
# queue_before_nvconv = Gst.ElementFactory.make("queue", "queue_before_nvconv")
if not nvvideoconvert or not videorate or not capsfilter or not nvvideoconvert2 or not capsfilter2 or not queue:
sys.stderr.write(" Unable to create elements for source bin \n")
caps = Gst.Caps.from_string("video/x-raw(memory:NVMM),framerate=25/1")
capsfilter.set_property("caps", caps)
caps2 = Gst.Caps.from_string("video/x-raw(memory:NVMM),format=RGBA")
capsfilter2.set_property("caps", caps2)
videorate.set_property("max-rate", 25)
videorate.set_property("drop-only", False)
videorate.set_property("max-duplication-time", 20000000)
# Add elements to the bin
nbin.add(uri_decode_bin)
# nbin.add(queue_before_nvconv)
nbin.add(nvvideoconvert)
nbin.add(videorate)
nbin.add(nvvideoconvert2)
nbin.add(capsfilter)
nbin.add(capsfilter2)
nbin.add(queue)
# Link elements
uri_decode_bin.link(nvvideoconvert)
nvvideoconvert.link(videorate)
videorate.link(nvvideoconvert2)
nvvideoconvert2.link(capsfilter2)
capsfilter2.link(capsfilter)
# nvvideoconvert2.link(capsfilter)
capsfilter.link(queue)
# We need to create a ghost pad for the source bin which will act as a proxy
# for the video decoder src pad. The ghost pad will not have a target right
# now. Once the decode bin creates the video decoder and generates the
# cb_newpad callback, we will set the ghost pad target to the video decoder
# src pad.
bin_pad = nbin.add_pad(Gst.GhostPad.new_no_target("src", Gst.PadDirection.SRC))
if not bin_pad:
sys.stderr.write(" Failed to add ghost pad in source bin \n")
return None
return nbin
The videorate will change the timestamps, which may also introduce some bias. If the input sources are all 10 FPS, the “streammux.set_property(“batched-push-timeout”, 40000)” should be “streammux.set_property(“batched-push-timeout”, 100000)”.
What is the nvinfer batch size?
it didnt help, but changing live-source from false to true made it worse, batch-size is 30, that is the maximum channels we need to run.
There should be some delay between RTSP + decoder and inferencing
in the rtspsrc?
The latency between inferencing and OSD did not help.
inference is happening in a different container. And latency is introduced in a different container, in rtspsrc.
my pipeline at overlay container looks like this
gst-launch-1.0 -e rtspsrc location=rtsp://192.168.31.4:8555/video1 latency=600 drop-on-latency=true ! decodebin ! nvvideoconvert ! videorate max-duplication-time=20000000 drop-only=false max-rate=25 ! 'video/x-raw,framerate=25/1' ! nvvideoconvert ! m.sink_0 nvstreammux batch-size=2 width=1280 height=720 live-source=1 name=m batched-push-timeout=40000 ! nvstreamdemux name=d d.src_0 ! tee name=t ! queue name=delay_queue ! autovideosink sync=false
Your inferencing includes the RTSP+decoding. They should be separated.
This pipeline is for display. The “drop-on-latency=true” setting will make the artifacts worse.
What we say “inferencing” means “nvstreammux+nvinfer/nvinferserver”
is a must, and without it after some time the buffering will increase the latency to 10s of seconds. We need real time suvellience.
we are inferencing using nvinferserver with python backend on a triton inference server framework, Full GPU-GPU
