Nvarguscamerasrc timestamp from buffer metadata JP 4.4 (3 cameras sinchronized)

Autonomous Machines Jetson & Embedded Systems Jetson TX2
#1

Hi all,

In the past using JetPack 3.2.1 I was able to get the metadata from the nvbuffer using nvcamerasrc and the property enable-meta=true, for three cameras synchronized we were receiving 3 same timestamps (meta->timestamp from nvcamerasrc)

>>> Gstreamer: sensor-0 Frame #257 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603478560
>>> Gstreamer: sensor-2 Frame #259 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603431560
>>> Gstreamer: sensor-1 Frame #261 : Timestamp: 46262843 - Trigger: 549737122944 - Diff: 18446698360603455560

So in JetPack 3.2.1 everything is working as expected, I expect 3 same timestamps when the cameras are synchronized.

The problem is when doing the same test but in JetPack 4.4 in this case using nvarguscamerasrc (enable-meta is not longer available, it is enable by default). we are not getting 3 same timestamps when the cameras are synchronized.

This is the setup for JetPack 4.4 where we are seeing the issue:

  • Same Hardware.
  • Same GStreamer application (the only change is nvcamerasrc by nvarguscamerasrc).
  • Same camera driver.
  • JetPack 4.4 instead JetPack 3.2.1
  • We have validated that the cameras are syncronized.
>>> Gstreamer:sensor-0 Frame #85 : Timestamp: 93789 - Trigger: 549706389008 - Diff: 455916640008
>>> Gstreamer: sensor-1 Frame #72 : Timestamp: 93793 - Trigger: 549706389008 - Diff: 455912659008
>>> Gstreamer:sensor-2 Frame #44 : Timestamp: 93801 - Trigger: 549706389008 - Diff: 455905319008
>>> Gstreamer:sensor-0 Frame #86 : Timestamp: 93822 - Trigger: 549706389008 - Diff: 455883467008
>>> Gstreamer: sensor-1 Frame #73 : Timestamp: 93827 - Trigger: 549706389008 - Diff: 455879282008
>>> Gstreamer:sensor-2 Frame #45 : Timestamp: 93832 - Trigger: 549706389008 - Diff: 455874303008
>>> Gstreamer:sensor-0 Frame #87 : Timestamp: 93856 - Trigger: 549706389008 - Diff: 455850061008
>>> Gstreamer: sensor-1 Frame #74 : Timestamp: 93860 - Trigger: 549706389008 - Diff: 455846164008
>>> Gstreamer:sensor-2 Frame #46 : Timestamp: 93866 - Trigger: 549706389008 - Diff: 455840272008
>>> Gstreamer:sensor-0 Frame #88 : Timestamp: 93890 - Trigger: 549706389008 - Diff: 455815645008
>>> Gstreamer: sensor-1 Frame #75 : Timestamp: 93893 - Trigger: 549706389008 - Diff: 455812869008
>>> Gstreamer:sensor-2 Frame #47 : Timestamp: 93898 - Trigger: 549706389008 - Diff: 455808021008
>>> Gstreamer:sensor-0 Frame #89 : Timestamp: 93933 - Trigger: 549706389008 - Diff: 455772801008
>>> Gstreamer:sensor-2 Frame #48 : Timestamp: 93937 - Trigger: 549706389008 - Diff: 455768815008
>>> Gstreamer: sensor-1 Frame #76 : Timestamp: 93941 - Trigger: 549706389008 - Diff: 455764970008
>>> Gstreamer:sensor-0 Frame #90 : Timestamp: 93956 - Trigger: 549706389008 - Diff: 455750130008
>>> Gstreamer: sensor-1 Frame #77 : Timestamp: 93960 - Trigger: 549706389008 - Diff: 455745948008
>>> Gstreamer:sensor-2 Frame #49 : Timestamp: 93964 - Trigger: 549706389008 - Diff: 455741460008
>>> Gstreamer:sensor-0 Frame #91 : Timestamp: 93989 - Trigger: 549706389008 - Diff: 455716922008
>>> Gstreamer: sensor-1 Frame #78 : Timestamp: 93993 - Trigger: 549706389008 - Diff: 455712625008
>>> Gstreamer:sensor-2 Frame #50 : Timestamp: 93998 - Trigger: 549706389008 - Diff: 455708068008
>>> Gstreamer:sensor-0 Frame #92 : Timestamp: 94024 - Trigger: 549706389008 - Diff: 455682255008

This is the code that we are using in the GStreamer application:

static GQuark gst_buffer_metadata_quark = 0;
typedef struct AuxBufferData {
  gint64 frame_num;
  gint64 timestamp;
} AuxData;
static GstPadProbeReturn
nvargus_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    AuxData *meta = NULL;
    GstBuffer *buf = (GstBuffer *) info->data;
    gst_buffer_metadata_quark = g_quark_from_static_string ("GstBufferMetaData");
    meta = (AuxData *) gst_mini_object_get_qdata (GST_MINI_OBJECT_CAST (buf),
							   gst_buffer_metadata_quark);
   .......
   printf(">>> Gstreamer:sensor-%d Frame #%lu : Timestamp: %lu - Trigger: %lu - Diff: %lu\n", sensor-id, meta->frame_num, meta->timestamp/1000000, trigger_ts, diff);
   return GST_PAD_PROBE_OK;
}

Could it be related to nvarguscamerasrc element?

Thanks,
-Adrian

#4

hello ACervantes,

please note that there’re hardware timestamp and software timestamps.
you’re should enable hardware frame-sync pin to output identical frames for your multiple camera setups, please also refer to Topic 78821 for checking sensor timestamps.
thanks

#5

Hi JerryChang,

thank you for your response, it seems that implementing getSensorTimestamp() at nvarguscamerasrc we are able to see the three timestamps from the sensors sinchronized.

>>> Gstreamer:sensor-2 Frame #39 : Timestamp: 355277811 - Trigger: 549047073184 - Diff: 18446389344944794800
>>> Gstreamer: sensor-1 Frame #68 : Timestamp: 355277811 - Trigger: 549047073184 - Diff: 18446389344944794800
>>> Gstreamer:sensor-0 Frame #92 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer:sensor-2 Frame #40 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer: sensor-1 Frame #69 : Timestamp: 355277845 - Trigger: 549047073184 - Diff: 18446389344911498800
>>> Gstreamer:sensor-0 Frame #93 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer:sensor-2 Frame #41 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer: sensor-1 Frame #70 : Timestamp: 355277878 - Trigger: 549047073184 - Diff: 18446389344878202800
>>> Gstreamer:sensor-2 Frame #42 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800
>>> Gstreamer:sensor-0 Frame #94 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800
>>> Gstreamer: sensor-1 Frame #71 : Timestamp: 355277911 - Trigger: 549047073184 - Diff: 18446389344844906800

Thank you again,
-Adrian