Smart Record Clips Show Grey Frames when inference performance drops

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson Xavier NX Developer Kit
• DeepStream Version
• JetPack Version (valid for Jetson only)
4.5.2 [L4T 32.5.1]
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

We are running inference on RTSP streams, and when certain objects are detected, we trigger smart record to record a clip of the RTSP stream.

This was working without issue with our initial model, with the pipeline FPS running at 5.07FPS.

We deployed a new model, which reduced the pipeline FPS to 4.23FPS. The smart record video clips for that camera then had an issue with grey frames - the video would go grey for periods throughout the video.

It is my understanding that the smart record call runs separately to the computer vision pipeline, and so increased inference run-time should not affect the smart record performance.

Our specific questions:

  • Is smart record performance/video quality affected by inference/pipeline runtime
  • If so, is there a way to handle this ?

Thanks in advance.

Hi,
Are you able to try DeepStream 6? Looks like you are on DeepStream 5. If it is possible, we woudl sgueest upgrade to latest version and give it a try.

Hi DaneLLL,

For the moment we are restricted to DeepStream 5 and cannot update to DeepStream 6. Is this a known issue in DeepStream 5 ?

Hi,
There is a topic similar to this:
Smart Record issue - record files have strange grey blocks in the middle

Please check if you observe the same symptom. We have bug fixes from DS 5 to DS 6. If you have another Xavier NX devkit, Would suggest flash Jetpack 4.6 + DS 6 for a try.

Hi,

We will upgrade our system to DS6 and report back with the results.

Hi,

We have upgraded to Deepstream 6 and run a more thorough experiment:

Source: RTSP feed from an NVR. 25 fps
Ffprobe output: Stream #0:0: Video: hevc (Main), yuvj420p(pc), 2688x1520 [SAR 1:1 DAR 168:95], 25 tbr, 90k tbn, 90k tbc

Our deepstream app is deployed in a container based on: nvcr.io/nvidia/deepstream-l4t:6.0-samples

Setup 1: Pedestrian Only Model with Fast Inference Speeds:

  • Pipeline runs at 24fps
  • No grey artifacts from smart record clips. All high quality. Sample screenshot attached

Setup 2: Multi-class model with slow inference speed:

  • Pipeline runs at 3fps
  • Grey artifacts observed on almost all cmart record clips. Sample screenshot attached.

We ran an additional experiment, where from the cli (on host device, outside the container) we recorded multiple clips from the RTSP stream with ffmpeg, for 3 cases:

  • deepstream app with slow model
  • deepstream app with fast model
  • no deepstream app deployed

and in each case the video clips were all artifact free with no grey blocks.