Program freezes when running python bindings created for body-pose-net-post-processing using example tao app

I am trying to create python bindings for deepstream 6.1 using the code in the github repo nvidia’s deepstream tao apps - bodyposenet app.

I exported a modified version of this function using pybind11 as follows:

The function prototype in nvidia’s code is:

static GstPadProbeReturn
pgie_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data);

I created a new function whose prototype is:

static GstPadProbeReturn
pgie_pad_buffer_probe_for_export (size_t gst_buffer)

so that, after getting the python bindings, the function can be called from python as follows:

# the following two lines are inside a probe function which gets "info" as an argument
def pgie_pad_buffer_probe(self, pad, info, u_data):
    gst_buffer = info.get_buffer()
    ... # checking the validity of gst_buffer here
    bodyposebindingsmodule.pgie_pad_buffer_probe_for_export( hash(gst_buffer) )  # the hash gives the c address

Comparison of nvidia’s pgie_pad_buffer_probe and my pgie_pad_buffer_probe_for_export:

In nvidia’s pgie_pad_buffer_probe, gst buffer is accessed as follows:

NvDsBatchMeta *batch_meta =
      gst_buffer_get_nvds_batch_meta (GST_BUFFER (info->data));

and wherever, gst_buffer is required, it was accessed as

GST_BUFFER (info->data)

In my pgie_pad_buffer_probe_for_export, those are modified as follows:

// we get the c address of gst buffer as input to this function as size_t, so we cast it
auto *buffer = reinterpret_cast<GstBuffer *>(gst_buffer);

and wherever GST_BUFFER (info->data) was there, it was replaced with the above buffer


The code for creating python bindings using pybind11 is:

PYBIND11_MODULE(bodyposebindingsmodule, m) {
   m.doc() = "pybind11 plugin for bodypose"; // optional module docstring

   m.def("pgie_pad_buffer_probe_for_export", &pgie_pad_buffer_probe_for_export, "The pgie_pad_buffer_probe function that calls bodypose postprocessor");
}

The compilation commands are:

c++ -O3 -Wall -shared -std=c++11 `pkg-config --cflags gstreamer-1.0` -I /opt/nvidia/deepstream/deepstream/sources/includes/ -I /opt/nvidia/deepstream/deepstream/sources/includes/cvcore_headers/ -I /usr/local/cuda-11.7/include -I ../common/ -I /opt/paralaxiom/venv/forpybind11/lib/python3.8/site-packages/pybind11/include/ -I /usr/include/python3.8 `pkg-config --cflags glib-2.0`  -fPIC $(python3 -m pybind11 --includes) ../common/ds_yml_parse.cpp `pkg-config --libs gstreamer-1.0` `pkg-config --libs glib-2.0` -L /opt/nvidia/deepstream/deepstream/lib/ -lnvdsgst_meta -lnvds_meta -lnvds_inferutils -lnvds_utils -lm -lstdc++ -lgstrtspserver-1.0 -lnvds_yml_parser -L/opt/nvidia/deepstream/deepstream/lib/cvcore_libs -L/usr/local/cuda-11.7/lib64/ -lcudart -lcuda -lyaml-cpp -lnvcv_bodypose2d -lnvcv_core -lnvcv_tensorops -lnvcv_trtbackend -Wl,-rpath,/opt/nvidia/deepstream/deepstream/lib/ -o ds_yml_parse.so
c++ -O3 -Wall -shared -std=c++11 `pkg-config --cflags gstreamer-1.0` -I /opt/nvidia/deepstream/deepstream/sources/includes/ -I /opt/nvidia/deepstream/deepstream/sources/includes/cvcore_headers/ -I /usr/local/cuda-11.7/include -I ../common/ -I /opt/paralaxiom/venv/forpybind11/lib/python3.8/site-packages/pybind11/include/ -I /usr/include/python3.8 `pkg-config --cflags glib-2.0`  -fPIC $(python3 -m pybind11 --includes) deepstream_bodypose2d_meta.cpp `pkg-config --libs gstreamer-1.0` `pkg-config --libs glib-2.0` -L /opt/nvidia/deepstream/deepstream/lib/ -lnvdsgst_meta -lnvds_meta -lnvds_inferutils -lnvds_utils -lm -lstdc++ -lgstrtspserver-1.0 -lnvds_yml_parser -L/opt/nvidia/deepstream/deepstream/lib/cvcore_libs -L/usr/local/cuda-11.7/lib64/ -lcudart -lcuda -lyaml-cpp -lnvcv_bodypose2d -lnvcv_core -lnvcv_tensorops -lnvcv_trtbackend -Wl,-rpath,/opt/nvidia/deepstream/deepstream/lib/ ds_yml_parse.so -o deepstream_bodypose2d_meta.so
c++ -O3 -Wall -shared -std=c++11 `pkg-config --cflags gstreamer-1.0` -I /opt/nvidia/deepstream/deepstream/sources/includes/ -I /opt/nvidia/deepstream/deepstream/sources/includes/cvcore_headers/ -I /usr/local/cuda-11.7/include -I ../common/ -I /opt/paralaxiom/venv/forpybind11/lib/python3.8/site-packages/pybind11/include/ -I /usr/include/python3.8 `pkg-config --cflags glib-2.0`  -fPIC $(python3 -m pybind11 --includes) deepstream_bodypose2d_app.cpp `pkg-config --libs gstreamer-1.0` `pkg-config --libs glib-2.0` -L /opt/nvidia/deepstream/deepstream/lib/ -lnvdsgst_meta -lnvds_meta -lnvds_inferutils -lnvds_utils -lm -lstdc++ -lgstrtspserver-1.0 -lnvds_yml_parser -L/opt/nvidia/deepstream/deepstream/lib/cvcore_libs -L/usr/local/cuda-11.7/lib64/ -lcudart -lcuda -lyaml-cpp -lnvcv_bodypose2d -lnvcv_core -lnvcv_tensorops -lnvcv_trtbackend -Wl,-rpath,/opt/nvidia/deepstream/deepstream/lib/ ds_yml_parse.so deepstream_bodypose2d_meta.so -o bodyposebindingsmodule$(python3-config --extension-suffix)

After importing the module in python deepstream, and when running, after the function is called, the program freezes (i.e. no statements printed, and the terminal keeps waiting).

Note: For this project, the repo trt_pose cannot be used (as suggested in some older forums on this topic)

Changing to Deepstream forum.

could you narrow down this issue by the following steps?

  1. if not calling pgie_pad_buffer_probe_for_export, will the app hang?
  2. if add log in pgie_pad_buffer_probe_for_export, will the app print the log?
  3. if empty pgie_pad_buffer_probe_for_export and calling an this empty function, will the app hang?

Thanks for replying fanzh.

The answers for the three questions in the same order are:

  1. No, the app did not hang in this case.
  2. Yes. I have cout statements in the exported function, which are being printed in the terminal. However, the print statement at the start of the function is printed but not the one at the end. I will put more cout statements in this function and see if the error location can be found.
  3. No, the app did not hang in this case too. However, thanks to this, I found a TypeError in python. The return type of the exported function was not being recognized in Python (even though, without the empty function, this point was not reached)

Please keep this issue open for now. I will collect some more information.

The cause of the freeze is that body2Dposepost which is of type cvcore::bodypose2d::BodyPose2DPostProcessor was being initialized in the main function in the cpp code. So, it was never initialized in the exported python function. Copying the relevant part from main function to the exported function solved it.

However, ran into a different error which says RuntimeError: invalid input pafmap dimension.. This post says that they solved it by restarting. I will try few things, and come here if error persists.

Thanks for the sharing.

1 Like

The cause of the second issue RuntimeError: invalid input pafmap dimension was also due to same issue.

Two variables cvcore::bodypose2d::BodyPose2DPostProcessorParams postProcessParams and cvcore::ModelInputParams ModelInputParams were being used while initializing body2Dposepost. The batchsize was different in the example app which is received from ModelInputParams, and also a data member named jointEdges of postProcessParams was being filled in the main function. Initializing postProcessParams and ModelInputParams with required values and using them in the intialization body2Dposepost made my program run without freezes and errors.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.