Delay in rendering live source compared to source

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only): 5.1.5
• TensorRT Version: 8.5.2.1
• NVIDIA GPU Driver Version (valid for GPU only) n/a
• Issue Type( questions, new requirements, bugs) bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Hi, I am running a custom Deepstream application based on the Deepstream reference app. I have an HDMI capture card connected that is connected to a video input source (in this case just another computer playing videos). the Deepstream is running as a live-source inferencer on the video input. I have the following strange behaviour:

  • If I start the application with the input source plugged into the HDMI input, I have a small delay (roughly 0.5 to 1 second) between the input and what is shown on the Jetson OSD. If I unplug and re-plug the input, the delay disappears.
  • If I start the application without an input source plugged into the HDMI input, and then plug it in once the application is running, there is also no delay.

If I set the FPS rate at 30 instead of 60, the delay disappears. However, I would like my application to run at 60FPS; and the scenarios above show that it is at least feasible.

I figured it may be something with timestamp/buffering issues, but can’t seem to find a fix. Please find below my configuration file:

[source0]
enable=1
type=1
camera-width=1920
camera-height=1080
camera-fps-n=60
camera-fps-d=1
camera-v4l2-dev-node=0

video-format=NV12
nvbuf-memory-type=0

[sink0]
enable=1
#Sink type: DRM is 5, windowed is 2
type=2
source-id=0
codec=1
sync=0
gpu-id=0
nvbuf-memory-type=0
conn-id=0

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-color=1;0;0;0
nvbuf-memory-type=0
process-mode=1
display-bbox=1
display-text=1

[streammux]
gpu-id=0
live-source=1
buffer-pool-size=1
sync-inputs=0
batch-size=1
batched-push-timeout=16666
width=1920
height=1080
enable-padding=0
max-latency=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=2
nvbuf-memory-type=0
bbox-border-color0=1;1;1;1
config-file=/path/to/config/file

[secondary-gie]
enable=1
gie-unique-id=3
operate-on-gie-id=2
config-file=/path/to/sgie/config/file

Any help would be appreciated. Thank you

Did you do the “unplug and re-plug the input” when the DeepStream pipeline is running?

Yes, the “unplug and re-plug the input”" is done whilst the Deepstream pipeline is running

The deepstream-app sample will exit if the camera is unplugged, how did you do that?

The HDMI capture card is an integrated PCIE card. When unplugging the HDMI cable, the deepstream app shows the following screen (but does not exit):

Can the issue be reproduced without any DeepStream component? We don’t have your device, we need your help to identify the root cause.

Hi Fiona,

I could not reproduce the issue outside of Deepstream.

As a temporary solution, I destroy and recreate the sink at application startup, which removes the delay. This will do for now.