RTSP output not visible for BodyPose2D DeepStream TAO app

• Jetson Xavier NX
• DeepStream 6.3
• JetPack 5.1.2
• TensorRT 8.5 (default that came after flashing)

Hello, good day.

I would like to use the BodyPose2D app to run inference on rtsp streams. I first deployed it on DeepStream 6.0, using the appropriate 6.0 branch tag on the DeepStream TAO Apps repo. This earlier version does not have rtsp capabilities, so I introduced the relevant rtsp server and appropriate elements and configuration into the app’s pipeline to run rtsp feeds. I could not get it to display video on VLC like I’ve been able to with my other DeepStream projects’ pipelines. I re-flashed my Jetson with the latest Jetpack and DeepStream that it could support (version details above) and deployed the appropriate version of the BodyPose2D app on the master branch of the same Github repo. I use both options to run the app: yaml file and the terminal flags. It seems to stream when I check it on VLC; the timer bar advances, but I still can’t get the rtsp output’s display. The app works fine on pre-recorded videos though, that is, saved video files. What else I’ve tried:

  1. I’ve downloaded the required Gstreamer rtsp server libraries
  2. I’ve verified through ffprobe and the UI of my web app that my rtsp feeds are up
  3. I’ve also tried different vacant/available port numbers for the udp and rtsp ports

What might I need to do further to get an output video feed?

Thanks for any help you can lend me :)

Hey @junshengy, any chance you’re familiar with this app?

You can refer to this sample

./apps/tao_others/deepstream-pose-classification/deepstream_pose_classification_app.cpp

Then merge the following code into BodyPose2D .

  } else if(sink_type == 2) {
    sink = gst_element_factory_make("nvrtspoutsinkbin", "nv-rtspsink");
    if (!sink) {
      g_printerr ("Filesink could not be created. Exiting.\n");
      return -1;
    }

Hey @junshengy, thanks
I applied the suggested fix in this section that creates the sink depending on the output type (in BodyPose2D, “4” is used for rtsp output):

else if (output_type == 4) {
    if (isImage) {
         ...}

    caps = ...;
    feature = ...;
    // sink = gst_element_factory_make ("udpsink", "nv-udpsink");
    sink = gst_element_factory_make("nvrtspoutsinkbin", "nv-rtspsink");
    if (!sink) {
      g_printerr ("Filesink could not be created. Exiting.\n");
      return -1;
    }
  }

The app exited with the designated error code showing the suggested sink is not accepted:

george@jetson:~/deepstream_tao_apps/apps/tao_others/deepstream-bodypose2d-app$ ./deepstream-bodypose2d-app ./bodypose2d_app_config.yml
pgie_type 0
Request sink_0 pad from streammux
Request sink_1 pad from streammux
Filesink could not be created. Exiting.

With the previous sink (udpsink) the app could build the pipeline and run. Is there something I’m missing?

Is there a DeepStream app suitable for my Jetson setup (Jetpack 5.1.2, DeepStream 6.3 with rtsp input sources) that NVIDIA proposes as best used for detecting body poses, to aid in classifying actions like running and walking, and especially falling?
I don’t strictly need to use BodyPose2D if there is a better pose-detection app out there, as long as it readily works. Hi @Fiona.Chen, have any suggestions?

Thanks

Any update on this, @junshengy @Fiona.Chen?

1.reset your code, use the following patch

diff --git a/apps/tao_others/deepstream-bodypose2d-app/deepstream_bodypose2d_app.cpp b/apps/tao_others/deepstream-bodypose2d-app/deepstream_bodypose2d_app.cpp
index e6c0cd3..c76e5b8 100644
--- a/apps/tao_others/deepstream-bodypose2d-app/deepstream_bodypose2d_app.cpp
+++ b/apps/tao_others/deepstream-bodypose2d-app/deepstream_bodypose2d_app.cpp
@@ -1018,6 +1018,8 @@ main (int argc, char *argv[])
       ds_parse_rtsp_output(sink, server, factory, argv[1], "output");
     } else {
       g_object_set (G_OBJECT (outenc), "bitrate", 4000000, NULL);
+      g_object_set (G_OBJECT (outenc), "insert-sps-pps", true, NULL);
+      g_object_set (G_OBJECT (outenc), "idrinterval", 1, NULL);
       g_object_set (G_OBJECT (sink), "host", "224.224.255.255", "port",
       atoi(argv[3]), "async", FALSE, "sync", 0, NULL);
  1. Rebuild application, the run the program like the following command.
./deepstream-bodypose2d-app 4 ../../../configs/nvinfer/bodypose2d_tao/sample_bodypose2d_model_config.txt 5000 8554 "rtsp://your rtsp uri" out

3.Play the out put rtsp stream.

ffplay rtsp://"your Xavier ip ":8554/ds-out-avc

This example is for demonstration only, you can use your own trained model.

Thanks @junshengy, after applying your fix I can now view the rtsp feed on VLC.

Can I run the app using the yaml file? Even after including the two properties from your fix into the yaml, it gives warnings and does not display an rtsp output:

!! [WARNING] Unknown param found : type
!! [WARNING] Unknown param found : enc
...etc.

Thanks again

This requires you to parse the configuration file yourself

I’m guessing this is done using this ds_yml_parse.cpp file I found in the deepstream_tao_apps/apps/tao_others/common/ folder.

Thank you, @junshengy, the issue’s been resolved.