however when I disconnect one half of the metamux one half of the pipeline works.
below is an example where I have commented out the portion that connects one branch to the metamux pad to get it working.
can you please help. Also can you also have a look at why I seem to be not be able to create the pipeline graph.
I use
# Enable debugging
Gst.debug_set_active(True)
Gst.debug_set_default_threshold(5)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
# this seem to not create a full pipeline .dot file (this only creates the legend but stops short of creating the actual pipeline graph)
Gst.debug_bin_to_dot_file(pipeline, Gst.DebugGraphDetails.ALL, f"pipeline_{timestamp}")
I’ve already shared code above. Just click on the hyperlinks named “not_working” and “partially working” they are almost identical to except for the bit that is commented out.
More specifically that’s the part where the logging(non inference in this example) branch is connected to the ‘metamux’
However if I comment the processing branch connection to the metamux while leaving the logging branch connection intact it won’t work preserving the topology (what I mean there is because of the symmetry (ignoring linking order) two branches are essentially identical and if we assume this symmetry when we disconnect each link to the metamux the other should work (even in this failed state where when both connections should work simultaneously)
To make the logging side work I have to sever the connection from the tee (that the branches originate) to the processing branch (I use the word processing but for the example code I have detached all that because the behaviour is the same regardless)
So I think the issue is linking order related (whichever links first works if the metamux has only a single sink pad connected) and when the two sinks are connected to the metamux. I don’t get the display out but I suspect the pipeline is working(rolling) in some sense because when I set the debug level to 5 (as I’ve done in the shared code) I can see it responding to pause and play events.
Bottom line. I could be using the metamux wrong because I think if I resplit the branches with a stream-demux and remux the four streams with the single streammux it will potentially work fine.
As I understand each branch that has batched data (each batch is of batch size two in my case) gets connected to the metamux where the batch meta is merged (because I intend to put GIEs on both branches as some point) and a batch size 4 (the combination of two batch size 2 streams) stream is created (a unified batch of size 4 is created) that is forwarded to the multi stream tiler for display rendering.
Correct me please if I understood the mechanics of metamux incorrectly.
Hi fanz thanks a lot for taking the time to answer; I will try that I’m on the phone right now
Basically it’s four sources muxed together in a stream-mux —> tee (splits into two identical branches) —-> branch [ stream demux —> select any two sources —> mux the two sources again with a streammux]—> metamux (combines the two branches together) —> normal pipeline with a tiller (two rows, two cols) and an eglsink
I added queues in between but they didn’t change anything.
If you want a drawn diagram I can do that later when I’m infront of a computer
I have also made the code easier to test, now you should be able to just run it without adding any paths to sources (please run in a Jetson Xavier or so with a monotor attached)
(here I have disconnected the linkage from one branch to the metamux to demonstrate the pipeline is working)
Please let me know if you need more!
Cheers,
Ganindu.
Note: To pause press CTRL +C (SIGINT) once, second SIGINT will resume and the thirs SIGNIT will stop and exit the pipeline.
uncommenting lines 539 to 548 will engage both main branches to the metamux. (as shown in the diagram)
@fanzh I can see the problem with the diagram. it is seems a two surfaces are arrriving at the EGLsink? and videomux is not happening at the metamux the way I expected (as I mentioned when setting up the original question)?
In your code, if I enable this part code and disable this line code, the application also hung, why will this case hang? please refer to the pipeline 1.zip (235.2 KB).
[property]
enable=1
pts-tolerance=60000
[user-configs]
[group-0]
# src-ids-model-<model unique ID>=<source ids>
# mux all source if don't set it.
#src-ids-model-1=0;1
#src-ids-model-2=1;2
#src-ids-model-3=1;2
about linking demux (logging branch) request pad to the same branch’s stream muxer logging pad. I tried with both batch size 2 and 1 for the logging branch. it seems the pieline is linked each time but if I unlink (I assume you were trying to see if both branches worked if only one source was demuxed on the logging side (previously commented out to demonstrate working behaviour with only a single branched linked to the metamux )
Just to be clear I still get no dispay even with one source demuxed (when both branches are connected) I get a display only with without_logging_linked_tometamux.png see attached zip file
it goes to line 521: streamdemux_logging_srcpad_0.link(streammux_logging_sinkpad_0)
but the attached picture (in the zipfile)
I can see that the both pads in the demux to to mux is linked. but there is one unlinked pad on the metamux.
so here is my pipeline for that check metamux_sink_pad_created_but_not_linked.png
for the case where (line 521: streamdemux_logging_srcpad_0.link(streammux_logging_sinkpad_0 is commented please refer to with_two_metamux_sinkpads_both_branches_connected.png
apologies, I may have misunderstood you, what do you mean by hanging?
I never got any hanging in the conventional sense because pipelines rolled pretty much every time.
I say that because in all configurations it responds to pause/play events.
The problem is not getting a display.
In this particular case are you trying to see if it works with a created but unlinked metamux pad? (See the images I attached to the previous reply please)
In the images I attached in my previous reply can you point out which pipeline image you are you referring to (because I was not sure because the image you attached and the description was of two different things)
Based on your code, I added a probe function on metamux’s src, and added some printing in this function. hanging means the application did not print this log.
Based on your code, I unlinked queue_processing and metamux , then linked queue_logging and memamux, the application will hang. please refer to the code no.py (22.8 KB) and pipeline 1.zip above.
yes, if one branch is linked to metamux, the app still can run. you can use deepstream_parallel_inference_app to test. please copy cfg to configs/apps/bodypose_yolo. the test command is “./apps/deepstream-parallel-infer/deepstream-parallel-infer -c configs/apps/bodypose_yolo/source4_1080p_dec_parallel_infer.yml”. please refer to the log. source4_1080p_dec_parallel_infer.yml (6.6 KB) 11.30.txt (97.3 KB) pipeline.zip (678.8 KB)
Thanks a lot I will test that! (Sorry I did not have time to get to this today as I was setting up a prototype)
So the metamux can combine multiple streams together (not just the stream connected to the active pad) also can there be multiple active pads. E.g. sink_0;sink_1 ?
no there is only one active-pand, which means “Active sink pad which buffer will transfer to src pad”. please use gst-inspect-1.0 nvdsmetamux to check.
