Nvarguscamerasrc timestamp from buffer metadata JP 4.4 (3 cameras sinchronized)

Hi all,

In the past using JetPack 3.2.1 I was able to get the metadata from the nvbuffer using nvcamerasrc and the property enable-meta=true, for three cameras synchronized we were receiving 3 same timestamps (meta->timestamp from nvcamerasrc)

>>> Gstreamer: sensor-0 Frame #257 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603478560
>>> Gstreamer: sensor-2 Frame #259 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603431560
>>> Gstreamer: sensor-1 Frame #261 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603455560

So in JetPack 3.2.1 everything is working as expected, I expect 3 same timestamps when the cameras are synchronized.

The problem is when doing the same test but in JetPack 4.4 in this case using nvarguscamerasrc (enable-meta is not longer available, it is enable by default). we are not getting 3 same timestamps when the cameras are synchronized.

This is the setup for JetPack 4.4 where we are seeing the issue:

  • Same Hardware.
  • Same GStreamer application (the only change is nvcamerasrc by nvarguscamerasrc).
  • Same camera driver.
  • JetPack 4.4 instead JetPack 3.2.1
  • We have validated that the cameras are syncronized.
>>> Gstreamer:sensor-0 Frame #85 : Timestamp: 93789 - Trigger: 549706389008 - Diff: 455916640008
>>> Gstreamer: sensor-1 Frame #72 : Timestamp: 93793 - Trigger: 549706389008 - Diff: 455912659008
>>> Gstreamer:sensor-2 Frame #44 : Timestamp: 93801 - Trigger: 549706389008 - Diff: 455905319008
>>> Gstreamer:sensor-0 Frame #86 : Timestamp: 93822 - Trigger: 549706389008 - Diff: 455883467008
>>> Gstreamer: sensor-1 Frame #73 : Timestamp: 93827 - Trigger: 549706389008 - Diff: 455879282008
>>> Gstreamer:sensor-2 Frame #45 : Timestamp: 93832 - Trigger: 549706389008 - Diff: 455874303008
>>> Gstreamer:sensor-0 Frame #87 : Timestamp: 93856 - Trigger: 549706389008 - Diff: 455850061008
>>> Gstreamer: sensor-1 Frame #74 : Timestamp: 93860 - Trigger: 549706389008 - Diff: 455846164008
>>> Gstreamer:sensor-2 Frame #46 : Timestamp: 93866 - Trigger: 549706389008 - Diff: 455840272008
>>> Gstreamer:sensor-0 Frame #88 : Timestamp: 93890 - Trigger: 549706389008 - Diff: 455815645008
>>> Gstreamer: sensor-1 Frame #75 : Timestamp: 93893 - Trigger: 549706389008 - Diff: 455812869008
>>> Gstreamer:sensor-2 Frame #47 : Timestamp: 93898 - Trigger: 549706389008 - Diff: 455808021008
>>> Gstreamer:sensor-0 Frame #89 : Timestamp: 93933 - Trigger: 549706389008 - Diff: 455772801008
>>> Gstreamer:sensor-2 Frame #48 : Timestamp: 93937 - Trigger: 549706389008 - Diff: 455768815008
>>> Gstreamer: sensor-1 Frame #76 : Timestamp: 93941 - Trigger: 549706389008 - Diff: 455764970008
>>> Gstreamer:sensor-0 Frame #90 : Timestamp: 93956 - Trigger: 549706389008 - Diff: 455750130008
>>> Gstreamer: sensor-1 Frame #77 : Timestamp: 93960 - Trigger: 549706389008 - Diff: 455745948008
>>> Gstreamer:sensor-2 Frame #49 : Timestamp: 93964 - Trigger: 549706389008 - Diff: 455741460008
>>> Gstreamer:sensor-0 Frame #91 : Timestamp: 93989 - Trigger: 549706389008 - Diff: 455716922008
>>> Gstreamer: sensor-1 Frame #78 : Timestamp: 93993 - Trigger: 549706389008 - Diff: 455712625008
>>> Gstreamer:sensor-2 Frame #50 : Timestamp: 93998 - Trigger: 549706389008 - Diff: 455708068008
>>> Gstreamer:sensor-0 Frame #92 : Timestamp: 94024 - Trigger: 549706389008 - Diff: 455682255008

This is the code that we are using in the GStreamer application:

static GQuark gst_buffer_metadata_quark = 0;
typedef struct AuxBufferData {
  gint64 frame_num;
  gint64 timestamp;
} AuxData;
static GstPadProbeReturn
nvargus_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    AuxData *meta = NULL;
    GstBuffer *buf = (GstBuffer *) info->data;
    gst_buffer_metadata_quark = g_quark_from_static_string ("GstBufferMetaData");
    meta = (AuxData *) gst_mini_object_get_qdata (GST_MINI_OBJECT_CAST (buf),
							   gst_buffer_metadata_quark);
   .......
   printf(">>> Gstreamer:sensor-%d Frame #%lu : Timestamp: %lu - Trigger: %lu - Diff: %lu\n", sensor-id, meta->frame_num, meta->timestamp/1000000, trigger_ts, diff);
   return GST_PAD_PROBE_OK;
}

Could it be related to nvarguscamerasrc element?

Thanks,
-Adrian

hello ACervantes,

please note that there’re hardware timestamp and software timestamps.
you’re should enable hardware frame-sync pin to output identical frames for your multiple camera setups, please also refer to Topic 78821 for checking sensor timestamps.
thanks

Hi JerryChang,

thank you for your response, it seems that implementing getSensorTimestamp() at nvarguscamerasrc we are able to see the three timestamps from the sensors sinchronized.

>>> Gstreamer:sensor-2 Frame #39 : Timestamp: 355277811 - Trigger: 549047073184 - Diff: 18446389344944794800
>>> Gstreamer: sensor-1 Frame #68 : Timestamp: 355277811 - Trigger: 549047073184 - Diff: 18446389344944794800
>>> Gstreamer:sensor-0 Frame #92 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer:sensor-2 Frame #40 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer: sensor-1 Frame #69 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer:sensor-0 Frame #93 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer:sensor-2 Frame #41 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer: sensor-1 Frame #70 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer:sensor-2 Frame #42 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800
>>> Gstreamer:sensor-0 Frame #94 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800
>>> Gstreamer: sensor-1 Frame #71 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800

Thank you again,
-Adrian