DeepStream Pipeline Camera Sync Issue After Prolonged Running

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only) 5.1.3
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)

I have a DeepStream pipeline based on the deepstream-test3 sample application, running two object detectors:

  1. A wheel detection model
  2. A nut detection model (which only runs when a wheel is detected)

The pipeline is deployed on a Jetson Xavier and processes video from two rtsp cameras:

  • One camera captures the left wheel of a truck
  • The other captures the right wheel

Performance Observations

  • When no wheels are detected, the pipeline runs at around 15 FPS, since the nut detection model is inactive.
  • When wheels and nuts are detected, the FPS drops to 8-11 FPS (expected, as the second model starts processing).

To ensure real-time processing, I have set the camera FPS to 8, so the pipeline should not drop below 8 FPS under normal conditions. However, if FPS drops momentarily, the pipeline compensates by processing additional frames in the next few seconds to catch up. The logs below illustrate this behavior:

2025-03-18 03:23:02,225 : INFO: PID 1: PERF: {0: 8.0, 1: 8.0}
2025-03-18 03:23:22,227 : INFO: PID 1: PERF: {0: 8.0, 1: 8.0}
2025-03-18 03:23:42,228 : INFO: PID 1: PERF: {0: 8.0, 1: 8.0}
2025-03-18 03:24:02,229 : INFO: PID 1: PERF: {0: 8.0, 1: 8.0}
2025-03-18 03:24:22,230 : INFO: PID 1: PERF: {0: 6.65, 1: 6.65}
2025-03-18 03:24:42,231 : INFO: PID 1: PERF: {0: 9.35, 1: 9.35}

Issue: Camera Sync Loss Over Time

After running continuously for 3-4 days, the two camera feeds become unsynchronized.

  • My dashboard logs images with timestamps.
  • The left-side wheel is detected ~30 seconds earlier than the right-side wheel, even though both cameras started in sync.

This desynchronization increases over time, even though FPS fluctuations are balanced. Can you guide me how to fix this issue.