Add count display to "deepstream_app.c"

How can I use this code to count objects on the deepstream_app, the code is from “deepstream_test1_app.c”

static GstPadProbeReturn
osd_sink_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    GstBuffer *buf = (GstBuffer *) info->data;
    guint num_rects = 0; 
    NvDsObjectMeta *obj_meta = NULL;
    guint vehicle_count = 0;
    guint person_count = 0;
    NvDsMetaList * l_frame = NULL;
    NvDsMetaList * l_obj = NULL;
    NvDsDisplayMeta *display_meta = NULL;

    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);

    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
        int offset = 0;
        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
                l_obj = l_obj->next) {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);
            if (obj_meta->class_id == PGIE_CLASS_ID_VEHICLE) {
                vehicle_count++;
                num_rects++;
            }
            if (obj_meta->class_id == PGIE_CLASS_ID_PERSON) {
                person_count++;
                num_rects++;
            }
        }
        display_meta = nvds_acquire_display_meta_from_pool(batch_meta);
        NvOSD_TextParams *txt_params  = &display_meta->text_params[0];
        display_meta->num_labels = 1;
        txt_params->display_text = g_malloc0 (MAX_DISPLAY_LEN);
        offset = snprintf(txt_params->display_text, MAX_DISPLAY_LEN, "Person = %d ", person_count);
        offset = snprintf(txt_params->display_text + offset , MAX_DISPLAY_LEN, "Vehicle = %d ", vehicle_count);

        /* Now set the offsets where the string should appear */
        txt_params->x_offset = 10;
        txt_params->y_offset = 12;

        /* Font , font-color and font-size */
        txt_params->font_params.font_name = "Serif";
        txt_params->font_params.font_size = 10;
        txt_params->font_params.font_color.red = 1.0;
        txt_params->font_params.font_color.green = 1.0;
        txt_params->font_params.font_color.blue = 1.0;
        txt_params->font_params.font_color.alpha = 1.0;

        /* Text background color */
        txt_params->set_bg_clr = 1;
        txt_params->text_bg_clr.red = 0.0;
        txt_params->text_bg_clr.green = 0.0;
        txt_params->text_bg_clr.blue = 0.0;
        txt_params->text_bg_clr.alpha = 1.0;

        nvds_add_display_meta_to_frame(frame_meta, display_meta);
    }

    g_print ("Frame Number = %d Number of objects = %d "
            "Vehicle Count = %d Person Count = %d\n",
            frame_number, num_rects, vehicle_count, person_count);
    frame_number++;
    return GST_PAD_PROBE_OK;
}

Please take a look at nvtracker. The tracker element is meant to track unique objects across frames and you can keep count of objects like that. Deepstream test 2 has an example of how to use it.

You could just have a global counter, or one on u_data and add the xxx_count to that every iteration, but that probably isn’t what you want.

Yes I have use that tracker in other experiments, now what I want is to display that count like the “deepstream_test1_app.c” a permanent label for the objects instead of showing on top the bounding box.

I am not sure what you mean by a “permanant label”. Please clarify. You mean you wish to identify individual cars or people (not just count them) and track them across cameras? Something like that? So if somebody leaves the frame and comes back they’re not counted twice?

I want something like this example https://imgur.com/aq9mVvK
Show the count of the objects detected but I can’t make it work with the example code of my first comment.

That imgur link actually looks like a modified deepstream_test_1 with rtsp grafted on. Heh. Code gets around.

The issue with that example specifically is it only shows how many objects are in a frame (at a single moment in time). It can’t count, for example, how many cars pass the camera in an hour.

Deepstream test 2 can do that, however, by using a tracker.

You need a tracker element to let the pipeline know that a car in frame 1 is the same car in frame 2. By attaching this metadata to the buffer, it also lets the secondary inference engines know that they don’t have to do an inference on every object per frame… only once per unique object.

For example. the Primary inference engine detects cars and people. The primary inference engine attaches metadata on the buffer with bounding boxes. The tracker element then uses various methods to assign each unique object an id that remains the same even over multiple frames.

When the buffers and metadata hit the secondary inference engines, the secondary inferences are performed to figure out, for example, what make and model each car is. I recommend reading the deepstream plugin manual as it explains a lot of these details that aren’t always entirely clear from the examples.

As far as actually drawing to the display in C, the code for doing that is here in the example you posted. Basically you set display_text to what you want and when the metadata hits the osd element downstream, it draws this for you. Here is the same logic in Python if you prefer.

...
        display_meta = nvds_acquire_display_meta_from_pool(batch_meta);
        NvOSD_TextParams *txt_params  = &display_meta->text_params[0];
        display_meta->num_labels = 1;
        txt_params->display_text = g_malloc0 (MAX_DISPLAY_LEN);
        offset = snprintf(txt_params->display_text, MAX_DISPLAY_LEN, "Person = %d ", person_count);
        offset = snprintf(txt_params->display_text + offset , MAX_DISPLAY_LEN, "Vehicle = %d ", vehicle_count);

        /* Now set the offsets where the string should appear */
        txt_params->x_offset = 10;
        txt_params->y_offset = 12;

        /* Font , font-color and font-size */
        txt_params->font_params.font_name = "Serif";
        txt_params->font_params.font_size = 10;
        txt_params->font_params.font_color.red = 1.0;
        txt_params->font_params.font_color.green = 1.0;
        txt_params->font_params.font_color.blue = 1.0;
        txt_params->font_params.font_color.alpha = 1.0;

        /* Text background color */
        txt_params->set_bg_clr = 1;
        txt_params->text_bg_clr.red = 0.0;
        txt_params->text_bg_clr.green = 0.0;
        txt_params->text_bg_clr.blue = 0.0;
        txt_params->text_bg_clr.alpha = 1.0;

The image of that imgur link is from the deepstream_test_1 example that nvidia provides, I literally copy the code that display the counter into the main “deepstream_app” but it doesn’t work, I think other things have to be change.

Where are you pasting it? Have you added a probe of type BUFFER to the osd element’s sink pad? Becuase if you just add that code to some .c file, it’ll still need to be called, and by the right caller (a pad probe). An example of how that code is connected can be found in deepstream_test_1 here:

/* Lets add probe to get informed of the meta data generated, we add probe to
   * the sink pad of the osd element, since by that time, the buffer would have
   * had got all the metadata. */
  osd_sink_pad = gst_element_get_static_pad (nvosd, "sink");
  if (!osd_sink_pad)
    g_print ("Unable to get sink pad\n");
  else
    gst_pad_add_probe (osd_sink_pad, GST_PAD_PROBE_TYPE_BUFFER,
        <b>osd_sink_pad_buffer_probe</b>, NULL, NULL);

What that does is: every time that pad (osd_sink_pad) gets a buffer, it sends the buffer to the callback (osd_sink_pad_buffer_probe), which specifies the drawing operations / whatever, and sends the buffer and any attached metadata (what to draw, print, bounding boxes, etc…) on it’s way by returning GST_PAD_PROBE_OK. Other return types are possible if you want to, say, discard a buffer.

Yes, I copy/paste that code into deepstream_app.c, I don’t know how to call the probe because the code making the pipeline changes so much from the deepstream_test_1 to deepstream_app.c

I haven’t actually read the deepstream source, but a search for “probe” revealed this promising spot you might choose. You’ll have to remove the probe you add as well at some point by it’s id. Seems it’s done here. Somehow you’ll have to pass the probe_id around.

There are other things you will need to change as well, probably. You might consider starting from deepstream_test_1 and adding things instead of trying to modify deepstream_app.

If you do, i’d advise reading the whole thing, as well as the Gstreamer tutorials if you haven’t already since they cover things like pad probes and how to attach them. Warning: the freedesktop.org server is terrible, so Google cache is your friend.

Gstreamer is not a very friendly, idiomatic, modern, or bug free framework, but it is fast, and you’ll get out of it what you put into it. Very steep learning curve, but after half a year of looking at 30 charachter long functions, it’s stating to make some sense, at least to me.

1 Like