Hi. I’m currently working on building a parallel pipeline to distribute load and make the code more granular. My intention is to have two or more independent stages which will be merged after inference is done, so that the output of both is viewed on only one sink. Initially I though about using an NvStreamMux to merge the stages and then connect them to NvMultiStreamTile so that the output merges.
Here’s a diagram of the current working pipeline (Based on DeepStream Sample 2 and 3).
The idea is to make the following modification.
I implemented the pipeline using the Python bindings. Here’s the code implementation:
(We connect just one sink of the streammux, this would be later expanded to more inputs)
sgie2.link(sgie3)
# sgie3.link(tiler) # We bypass the existing link between sgie3 and tiler
##############################################################
# Create inference type mux
inferencemux = Gst.ElementFactory.make("nvstreammux", "Inference-muxer")
if not inferencemux:
sys.stderr.write(" Unable to create NvStreamMux \n")
pipeline.add(inferencemux)
inferencemux.set_property('width', 1920)
inferencemux.set_property('height', 1080)
inferencemux.set_property('batch-size', 2) # Is it 2?
inferencemux.set_property('num-surfaces-per-frame', 2)
inferencemux.set_property('batched-push-timeout', 4000000)
# Link the inferencemux[0] sink pad with the sgie3 src pad
padname = "sink_0"
sinkpad = inferencemux.get_request_pad(padname)
if not sinkpad:
sys.stderr.write("Unable to create sink pad bin \n")
srcpad = sgie3.get_static_pad("src")
if not srcpad:
sys.stderr.write("Unable to create src pad bin \n")
srcpad.link(sinkpad)
inferencemux.link(tiler)
###############################################################
# pgie.link(tiler) # Old flow
tiler.link(nvvidconv)
The pipeline builds correctly but fails we running after some seconds. Here are some of the errors:
0:00:06.080790998 27624 0x7f140131e0 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:06.080909075 27624 0x7f140131e0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f1801f380 Failed to determine interlace mode
0:00:06.080791207 27624 0x7f20014400 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:<nvv4l2decoder1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
We do get some output on the tiler for a couple seconds, but no bounding boxes are generated.
Is this approach possible?