Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
Jetson Xavier NX + ConnectTech Quark
• DeepStream Version
6.0
• JetPack Version (valid for Jetson only)
4.6
• Issue Type( questions, new requirements, bugs)
Questions
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
Our goal is to get audio and video from 1 RTSP input source and get them synchronized.
Audio signal will come in. At that exact point, we want to grab the frame with same/similar timestamp.
And for 2 seconds after that point, we want to collect frames.
• What we are referencing and how we are testing
-
We set it up so we know we get synchronized stream (AV) - https://docs.nvidia.com/metropolis/deepstream/dev-guide/_static/Nvidia.ico)AV Sync in DeepStream — DeepStream 6.0.1 Release documentation](AV Sync in DeepStream — DeepStream 6.1.1 Release documentation
-
After that we receive receive an audio signal, we grab the frame with signal because that is the exact one that had audio signal change.
-
Since we are using Python version, we can’t get NTP timestamp (since
configure_source_for_ntp_sync
function isn’t available). But we can assign system time as ntp using #1 method shown in this doc - NTP Timestamp in DeepStream — DeepStream 6.1.1 Release documentation -
We then grab that image as 1st reference. And start saving, we wait until we get the frame whose timestamp is
triggered_timestamp
+ 2 sec
• Question
-
We want to understand if #1 above allows us to synchronize the sound and the video. In the doc, we saw a sample pipeline (AV Sync in DeepStream — DeepStream 6.1.1 Release documentation) but it seemed like it was grabbing 2 RTSP streams and creating 1 output. In our case, we are looking to get audio and video from 1 RTSP source, and make sure that we grab/find the right frame based on the time we received the audio signal
-
We tried a few things, but we seem be not getting the result we want, so we wanted to see if the community can provide some feedback/guidance.
Thank you