Smart Record isn't so smart

• Hardware Platform Jetson
• DeepStream Version 5.0
• JetPack Version 4.5
• TensorRT Version 7

Hi,

I’ve been doing some development with Deepstream and have successfully hooked up the smart record function to MIPI/USB cameras. That is not my complaint.

My issue is that the documentation for Smart Record doesn’t make clear there are some major limitations in Smart Record. Specifically, (and I’m guessing this is because it uses a Gstreamer Queue as it’s method of buffering) that once a smart record is started for a start value > 0 - i.e. in the past - you can not make a subsequent record for any time in the past until the queue fills up again to that time.

I’ll give an example to make it clear. I’m assuming that my smart record cache value is set suitably large (say, 300 seconds).

If at t0 I call NvDsSRStart with a start value of 30 and a duration of 1 (the minimum value, as 0 would keep recording until the maximum time is reached), it will make a recording of approximately 31 second per the documentation and I’ll get the recording complete callback soon after 1 second. This video will approximately cover t0-30 to t0+1 (give or take keyframes)

However, If, 15 seconds later at t0+15, I make another call to NvDsSRStart (and also making sure that ctx->recordOn is FALSE) with a start value of 30 and duration of 1, I will only get a recording of 15 seconds, covering from t0+1 to t0+16

This is because the internal queue in Smart Record is not a buffer that can be re-accessed - the frames are fed in to the encoder and then discarded.

I don’t believe this is adequately documented, and can only be inferred from looking at the Smart Record documentation (which shows a diagram of the queue), knowledge of GStreamer and the seemingly strange decision that actual recording duration be length of (start+duration) rather than just duration.

Worse, I don’t see any easy work around for this. In the use case I’m working with, recordings are delayed by 30 seconds while analysis takes place, then are triggered for 10 second bursts. I already have code to call ffmpeg to cut the video from t0-30 to t0-20 and whilst overlapping records could be feasibly captured (by manually controlling the end of recording then splitting the subsequent video file up with ffmpeg), in the case I describe above where two non-overlapping video requests are sent, there is nothing relevant available for the second request.

So, in summary, smart record can’t be used for the purpose it would seem to be intended for.

Any chance this could be worked on? Or if you open sourced it, (another bugbear of mine - being a poorly documented close sourced module means it’s a pain to find these things out) at least someone could replace the queue with a proper buffering system or modify such that the queue isn’t emptied with every start command - which would allow the overlapping method described above to work.

1 Like

Thank you for your information. It will take some time for us to review your comments.

We have acknowledge your feedback and we have added this requirement to our roadmap. The document will be improved to clarify the function and limitations too.