How to debug customized plugin and the full pipeline when it stucks

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson AGX Xavier Dev Kit
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only) 4.6
• TensorRT Version 8
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi guys,

I am writing my personal pipeline using deepstream-test5. I suppose to have 1 PGIE with a dsexample1 (Customized by me, opencv involved) and one SGIE followed by another dsexample2 (also opencv operation). I am using 2=EglSink 4=UDPSink 6=MsgConvBroker, and HLS Sink (I wrote) as well. My pipeline works well when I use EGL Sink only (display good). When I add (or use only) a udpsink or msgconvbroker, the pipeline runs for a frame (with everything detected, rabbitmq output) and then stucks. FPS turned to 0. When I am using deepstream-test5 with yolov3, all these sinks work well.
I would like to know if there is a debug method to find out what is the problem. Do i need to add some tees to sync the data? or deepstream-test5 just don’t work with MsgConvBroker + 2 dsexample? Do my dsexample1 and dsexample2 destroy the buffer? If so why EGL works really great.

Thank you all in advance

->PGIE->dsexample1->tracker->SGIE->dsexample2->…->MsgConvBroker/HLSSink/UDPSink (fail)

Best regards

to narrow down this isue, please check if it related to your modification, for example:
try ->PGIE->tracker->SGIE–>UDPSink,
try ->PGIE->dsexample1->tracker->SGIE->UDPSink,
try ->PGIE->dsexample1->tracker->SGIE-> dsexample2 ->UDPSink,

Hi Fan,

let me add more infos here. The pipeline is
streammux->PGIE->dsexample1->tracker->SGIE->dsexample2->osd->sink (ignore some of the plugins). I moved PGIE parse function to dsexample1 and SGIE parse function to dsexample2. I have done a lot of experiment in disabling some of the plugins (like tracker, dsexampleN, even SGIE. I also tried to comment some codes in deepstream-app (called by test5) and all of them died after 1st frame using Msgbroker (but I still get the results from MQ and EGL Sink of the 1st frame ).

I am not C/C++ expert so I usually use g_print() to help debugging the pipeline. I also tried GST_DEBUG=6 or even 7 but there wasn’t any information included. If my implementations of dsexamples are wrong then why I am getting outputs when using EGL SInk or RTSP output? If my implementation of Msgbroker is wrong then why I am getting the first frame result correctly?

I also think of moving dsexamples back to parse function in case there is something related that I don’t know.

Hope you can give my some guidance on these.

Thank you

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

  1. please refer to my last comment, “PGIE->tracker->SGIE–>UDPSink” is a basic pipeline, it should not stuck, then add the first dsexample, you can do nothing in dsexample to check if it will stuck, then add actual code in dsexample step by step to check if it will stuck.
    my aim is to check which code will trigger this hung issue.
  2. dsexample and all sinks are opensource, please add logs to debug, for example: add log in plugin entering function, and add log in plugin leaving function, then you can get the cost time.
    3 . please provide simplified code to reproduce this issue if possible, including configuration file.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.