Passing objects along with DeepStream

• Hardware Platform (Jetson / GPU) V100
• DeepStream Version 5.0 Developer Preview
• TensorRT Version 7.0.0.11

I’ve been building off of the sample deepstream-app (YOLOv3 object detector) to familiarize myself with the SDK.

Ultimately, I’m trying to do the following:

  1. Grab a system timestamp when a frame from a live video stream (maybe a RTSP source or V4L2 for a USB camera) arrives at the machine. This timestamp value is not used in the inference process nor given to the object detector, but I want to associate it with the output from the inference process for that specific frame, so I need to save it until inference finishes.
  2. Create a single network socket when the app starts up, and pass that socket in so that I can use it to send messages containing both inference output for a frame (bounding boxes, tracker IDs, etc.) and the aforementioned timestamp of that specific frame.

Basically I’m trying to figure out where in the code I should be grabbing this timestamp, where I should be creating my socket (in deepstream_app_main.c?), and how I should be passing the socket and timestamps through with the app so I can use them in other methods once the object detector and tracker have finished doing their thing.

Currently inside deepstream_app.c I see that analytics_done_buf_prob() calls write_kitti_track_output(), which I could modify to send socket messages instead of printing stuff to a file, but I can’t create the socket in there as it would end up getting created and destroyed for every frame of video.

Can you see test4 sample which can send messages to broker, like amqp, kafka, azure, other broker user could create, using deepstream messaging interface, nvds_msgapi, to implement a custom protocol message handler and integrate it with deepstream applications. It may meet your requiements.
for rtsp timestamp, you may refer to https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.14.html#wwpID0E3HA Part NTP Timestamp in DeepStream
You could refer to test4 README for a quick glance of nvmsgconv and nvmsgbroker plugins.
for more details, you could refer to documentation,
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.14.html#wwpID0E0VQ0HA
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.14.html#wwpID0E0VQ0HA Part Gst-nvmsgbroker

1 Like