Deepstream parallel inference failing to produce video output with 'nvmultistreamtiler'

Please provide complete information as applicable to your setup.

• Hardware Platform Jetson
• DeepStream Version 6.2
• JetPack Version 5.1.0

I am trying to test a python parallel pprocessing pipeline.

here is a simplified version of the parallel pipeline redced to the simplest possible version that I can replicate the issue/.


however when I disconnect one half of the metamux one half of the pipeline works.
below is an example where I have commented out the portion that connects one branch to the metamux pad to get it working.

can you please help. Also can you also have a look at why I seem to be not be able to create the pipeline graph.

I use

    # Enable debugging

    timestamp ="%Y%m%d_%H%M%S")

    # this seem to not create a full pipeline .dot file (this only creates the legend but stops short of creating the actual pipeline graph)
    Gst.debug_bin_to_dot_file(pipeline, Gst.DebugGraphDetails.ALL, f"pipeline_{timestamp}")

I ave mentiones the referenced I used above.


  1. noticing one code works while the other can’t work, could you share the whole media pipeline and highlight the difference?
  2. about refer to this topic for how to generate pipeline in python.

I’ve already shared code above. Just click on the hyperlinks named “not_working” and “partially working” they are almost identical to except for the bit that is commented out.

More specifically that’s the part where the logging(non inference in this example) branch is connected to the ‘metamux’

However if I comment the processing branch connection to the metamux while leaving the logging branch connection intact it won’t work preserving the topology (what I mean there is because of the symmetry (ignoring linking order) two branches are essentially identical and if we assume this symmetry when we disconnect each link to the metamux the other should work (even in this failed state where when both connections should work simultaneously)

To make the logging side work I have to sever the connection from the tee (that the branches originate) to the processing branch (I use the word processing but for the example code I have detached all that because the behaviour is the same regardless)

So I think the issue is linking order related (whichever links first works if the metamux has only a single sink pad connected) and when the two sinks are connected to the metamux. I don’t get the display out but I suspect the pipeline is working(rolling) in some sense because when I set the debug level to 5 (as I’ve done in the shared code) I can see it responding to pause and play events.

Bottom line. I could be using the metamux wrong because I think if I resplit the branches with a stream-demux and remux the four streams with the single streammux it will potentially work fine.

As I understand each branch that has batched data (each batch is of batch size two in my case) gets connected to the metamux where the batch meta is merged (because I intend to put GIEs on both branches as some point) and a batch size 4 (the combination of two batch size 2 streams) stream is created (a unified batch of size 4 is created) that is forwarded to the multi stream tiler for display rendering.

Correct me please if I understood the mechanics of metamux incorrectly.


could you use a diagram to illustrate the designed pipeline? can use sample deepstream_parallel_inference_app to reproduce this issue?

Hi fanz thanks a lot for taking the time to answer; I will try that I’m on the phone right now

Basically it’s four sources muxed together in a stream-mux —> tee (splits into two identical branches) —-> branch [ stream demux —> select any two sources —> mux the two sources again with a streammux]—> metamux (combines the two branches together) —> normal pipeline with a tiller (two rows, two cols) and an eglsink

I added queues in between but they didn’t change anything.

If you want a drawn diagram I can do that later when I’m infront of a computer

Hi Fanz,

Please find the picture¬

I have also made the code easier to test, now you should be able to just run it without adding any paths to sources (please run in a Jetson Xavier or so with a monotor attached)
(here I have disconnected the linkage from one branch to the metamux to demonstrate the pipeline is working)

Please let me know if you need more!


Note: To pause press CTRL +C (SIGINT) once, second SIGINT will resume and the thirs SIGNIT will stop and exit the pipeline.

uncommenting lines 539 to 548 will engage both main branches to the metamux. (as shown in the diagram)

graphs (dotfiles)
produces an image
does not produce an image

Note: some linking is not shown becase of dynamic linking (I will try to generate dotfile after setting pipeline state to PLAYING)

Note 2:

The pieline below is working (showing the top row in the tiler while bottom row is blank)

the pipeline below does not produce any video outpt at the eglsink

@fanzh I can see the problem with the diagram. it is seems a two surfaces are arrriving at the EGLsink? and videomux is not happening at the metamux the way I expected (as I mentioned when setting up the original question)?

Thanks for the sharing!

  1. all the screenshots are not clear, please share a zip file.
  2. how to run the code I will have a try.

HI Fanz,

I put the dotfiles so you can have the vector graphs

sudo apt-get install xdot and open the files.

the should run with no problems because I replaced file and camera sources with 4 testsources.

just put ./gst_debug_info/dotfiles and /gst_debug_info/debug_dump'or just rename the location to ./

os.environ["GST_DEBUG_DUMP_DOT_DIR"] = "./gst_debug_info/dotfiles"
os.putenv('GST_DEBUG_DUMP_DIR_DIR', './gst_debug_info/debug_dump')


  1. please share your metamux_config.txt.
  2. In your code, if I enable this part code and disable this line code, the application also hung, why will this case hang? please refer to the pipeline (235.2 KB).

Hi Fanz,

thans a lot for helping!!

config file



# src-ids-model-<model unique ID>=<source ids>
# mux all source if don't set it.

about linking demux (logging branch) request pad to the same branch’s stream muxer logging pad. I tried with both batch size 2 and 1 for the logging branch. it seems the pieline is linked each time but if I unlink (I assume you were trying to see if both branches worked if only one source was demuxed on the logging side (previously commented out to demonstrate working behaviour with only a single branched linked to the metamux )

Just to be clear I still get no dispay even with one source demuxed (when both branches are connected) I get a display only with without_logging_linked_tometamux.png see attached zip file (2.0 MB)

Please let me know if this makes sense.


EDIT: I used a test setup with two camera and two filesrcs instread testsrcs but everything else is pretty much the same.

@fanzh just realised something,

when you mentioned

it goes to line 521:

but the attached picture (in the zipfile)

I can see that the both pads in the demux to to mux is linked. but there is one unlinked pad on the metamux.
so here is my pipeline for that check metamux_sink_pad_created_but_not_linked.png

for the case where (line 521: is commented please refer to with_two_metamux_sinkpads_both_branches_connected.png (1.3 MB)

In both cases there was no display for me .

did you fix this issue? how did you fix?

Hi @fanz,

apologies, I may have misunderstood you, what do you mean by hanging?

I never got any hanging in the conventional sense because pipelines rolled pretty much every time.

I say that because in all configurations it responds to pause/play events.

The problem is not getting a display.

In this particular case are you trying to see if it works with a created but unlinked metamux pad? (See the images I attached to the previous reply please)

In the images I attached in my previous reply can you point out which pipeline image you are you referring to (because I was not sure because the image you attached and the description was of two different things)

Sorry if I got that wrong,


  1. Based on your code, I added a probe function on metamux’s src, and added some printing in this function. hanging means the application did not print this log.

  2. Based on your code, I unlinked queue_processing and metamux , then linked queue_logging and memamux, the application will hang. please refer to the code (22.8 KB) and pipeline above.

please set active-pad in metamux cfg because its default value is null.

Thanks will do! Is the active-pad for the metamux, so can it only pass the stream through one sink pad at a time?

I’m asking that for example is it sink_0 or sink_1
(If two sources are connected)

yes, if one branch is linked to metamux, the app still can run. you can use deepstream_parallel_inference_app to test. please copy cfg to configs/apps/bodypose_yolo. the test command is “./apps/deepstream-parallel-infer/deepstream-parallel-infer -c configs/apps/bodypose_yolo/source4_1080p_dec_parallel_infer.yml”. please refer to the log.
source4_1080p_dec_parallel_infer.yml (6.6 KB)
11.30.txt (97.3 KB) (678.8 KB)

Thanks a lot I will test that! (Sorry I did not have time to get to this today as I was setting up a prototype)

So the metamux can combine multiple streams together (not just the stream connected to the active pad) also can there be multiple active pads. E.g. sink_0;sink_1 ?

no there is only one active-pand, which means “Active sink pad which buffer will transfer to src pad”. please use gst-inspect-1.0 nvdsmetamux to check.