• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
6.1.0
I have a pipeline with nvstreammux -> nvinfer -> nvtracker -> nvdsanalytics -> hlssink2
I’m attaching a probe to nvtracker
to store in memory all object tracks inside each frame buffer.
self._gst_pipeline.get_by_name("tracker").get_static_pad("src").add_probe(
Gst.PadProbeType.BUFFER, self._add_pb_frame
)
...
def _add_pb_frame(self, pad, info):
...
pb_frame = pb.Frame()
for object_meta in get_object_meta(
pyds.gst_buffer_get_nvds_batch_meta(hash(buf))
):
pb_track = pb.Track()
pb_track.id = object_meta.object_id
pb_track.label = object_meta.obj_label
pb_track.bbox.x = object_meta.rect_params.left / frame_width
pb_track.bbox.y = object_meta.rect_params.top / frame_height
pb_track.bbox.w = object_meta.rect_params.width / frame_width
pb_track.bbox.h = object_meta.rect_params.height / frame_height
pb_frame.tracks.append(pb_track)
self._pb_frames.append(pb_frame)
return Gst.PadProbeReturn.OK
I also listen for splitmuxsink-fragment-closed
message (hlssink2
element). Every time I receive this message, I write all of the tracks to a file so that each 2 seconds of hls .ts
fragment, there would be a matching .data
file with all of the tracks for these 2 seconds.
Here is the code which writes pb_frame
to a file when receiving splitmuxsink-fragment-closed
message:
def _dump_segment_data(self, structure):
...
with open(data_file, "wb") as f:
for pb_frame in self._pb_frames:
f.write(struct.pack("i", pb_frame.ByteSize()))
f.write(pb_frame.SerializeToString())
...
print(len(self._pb_frames))
self._pb_frames.clear()
The issue is that if I have pipeline frame rate set to 15/1 and hls target duration set to 2 seconds for some reason first fragment receives 91 buffer probe calls (!) and then goes to “real-time” which is around 30 frames per 2sec = 15fps. It kinda pushes the data out of sync with the output video.
print(len(self._pb_frames)) <- from the previous code block
91
34
30
30
30
30
31
30
30
30
30
How can I sync nvinfer probe data with the output .ts fragments? Do you have any idea what am I doing wrong? Is nvinfer doing some internal buffering?
You can see what is the issue here. Yellow bboxes are from the data file.