Streammux changing PTS

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

GPU

• DeepStream Version

6.3

• JetPack Version (valid for Jetson only)
• TensorRT Version

Version from nvcr.io/nvidia/deepstream:6.3-gc-triton-devel

• NVIDIA GPU Driver Version (valid for GPU only)

NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0

• Issue Type( questions, new requirements, bugs)

question (bug?)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Add some handlers to print the pts:

static GstPadProbeReturn pts_probe1( GstPad          * pad,
                                                    GstPadProbeInfo * info,
                                                    gpointer          u_data )
{
    static int count = 0;

    GstBuffer       * buf          = ( GstBuffer * ) info->data;
    NvDsBatchMeta   * batch_meta = gst_buffer_get_nvds_batch_meta ( buf );
    GstClockTime   pts = GST_BUFFER_PTS ( buf );

    g_print ( "pts before [%d]:\t%ld\n", count, pts );

    count++;

    return GST_PAD_PROBE_OK;
}

static GstPadProbeReturn pts_probe2( GstPad          * pad,
                                                    GstPadProbeInfo * info,
                                                    gpointer          u_data )
{
    static int count = 0;

    GstBuffer       * buf          = ( GstBuffer * ) info->data;
    NvDsBatchMeta   * batch_meta = gst_buffer_get_nvds_batch_meta ( buf );
    GstClockTime   pts = GST_BUFFER_PTS ( buf );

    g_print ( "pts after  [%d]:\t%ld\n", count, pts );

    count++;

    return GST_PAD_PROBE_OK;
}

Add Probes before and after the streammux, as follows:

    GstPad * pad1 = gst_element_get_static_pad ( streammux, "sink_0" );
    if ( !pad1 )
    {
        g_print ( "Unable to get streammux pad\n" );
    }
    else
    {
        gst_pad_add_probe ( pad1, GST_PAD_PROBE_TYPE_BUFFER,
                            pts_probe1, NULL, NULL );
    }
    gst_object_unref ( pad1 );


    GstPad * pad2 = gst_element_get_static_pad ( pgie, "sink" );
    if ( !pad2 )
    {
        g_print ( "Unable to get pad2\n" );
    }
    else
    {
        gst_pad_add_probe ( pad2, GST_PAD_PROBE_TYPE_BUFFER,
                            pts_probe2, NULL, NULL );
    }
    gst_object_unref ( pad2 );

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi All,

I am trying to run a deepstream app that will save the inference results to a database for later retrieval. I am saving the pts value in the database so I can exactly match back the results, however I noticed that the values being saved are not the same when I then play the video back using gstreamer or opencv.

I have since traced this to the streammux plugin changing the pts. You can see below that after adding probes I get different pts timestamps before and after the the streammux plugin.

pts before [0]: 0
pts after  [0]: 0
pts before [1]: 66666666
pts after  [1]: 66666666
pts before [2]: 133334166   <---- Correct
pts after  [2]: 133333332     <---- Wrong!

I only do one video at a time, so naturally my thought was to remove the streammux plugin, but apparently the streammux is now a required plugin. Either way the pgie was not happy:

error: NvDsBatchMeta not found for input buffer.
ERROR from element primary-nvinference-engine: NvDsBatchMeta not found for input buffer.

So how do I stop streammux from messing with my pts timestamps?

2

To answer my own question, apparently I was looking at the batch pts, not the frame pts:


...

    //
    // Loop through the frames
    //
    for ( l_frame = batch_meta->frame_meta_list; l_frame != NULL;
          l_frame = l_frame->next )
    {
        NvDsFrameMeta * frame_meta = ( NvDsFrameMeta * ) (l_frame->data);
        
        g_print ( "pts: %ld\n", frame_meta->buf_pts );

...
4

Nvstreammux will create a new gstbuffer for batched frames, it will change the pts by the framerate. You can retrieve that by frame_num in NvDsFrameMeta structure.

5

Thanks for the answer - if I don’t want that behaviour then I have to stick with a single source, correct? And this is independent of the batching used by the pgie?

6

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

The deepstream pipeline can’t be run without nvstreammux.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.