The system processes data from two sources: a camera and a microphone. It generates 1-second video segments and saves them to the output_data folder. The goal is to save an AAC audio file only after the video H265 file have been successfully saved.
To complicate this a little bit lets assume that instead of 2 sinks there are 3 splitmuxsink components now and we do not know the file save order:
One for saving AAC audio files
One for saving H265 video files
One for saving PNG image files
The desired workflow and the order of saving files is as follows:
Save H265 video file
Save PNG video file
Save AAC audio file
How can I control the save order of files? I want to ensure H265 file is saved first, followed by PNG file. Only after both H265 and PNG files are complete should the AAC file be saved. Then the whole process repates as splitmusxinks are used, and every 1 second there are 3 files generated.
Could this be achieved with some GStreamer plugin or should I add a probe function to some elements in the pipeline (written in Python) that ensures the order?
No way to implement in pipeline level. Your pipeline is actually three pipelines which have no relationship. Your requirement needs detailed control on the low level raw data and timestamps. E.G. you need to know with which timestamp the H265 decoder should generate the key frame, so that you can save the h265 video from this timestamp and then you need to find out the pieces of audio which is aligned with the same timestamp. Seems you need to write your own muxer and file saver to handle the data order by yourself.
Is it possible to connect three different pipelines (two video and one audio) into a single pipeline and then use the probe function to decide which file is saved first?
I believe the new nvstreammux can concatenate multiple streams into one. Could this be the solution, or is writing custom plugins the only option?
Ok, Let’s assume now that I have a single pipeline with just video from file/camera source. This video is split using a tee element into two branches.
One branch saves the video in H264 format, while the other saves it in H265 format. Both branches perform different operations, such as inferences, detections or genreal video operations.
How can I control the file save order of the two pipelines that originate from the common pipeline?
I’ve told you in previous post. You can only control by yourself, no open source plugin meet your requirement. You can implement probe function or plugin to handle by yourself.
and I need to control the file save order (e.g., ensuring the video is always saved before the audio). The solution is to create a custom plugin for audio and video sinks, or add a probe function with variables to control the file save order?
If probe function could solve this problem, should it be connected to the ogg-demuxer in this case or to the sink pads of audio and video sink?
The suggestion is to write a mux+file saving sink plugin to handle the low level data processing by yourself. You can’t change the saving order of the files by any public sink plugins, most sinks work with clock( GstBaseSink (gstreamer.freedesktop.org)), and try to consume the data ASAP. If you want to break the clock and change the order, you need to hold all GstBuffers in one place and check which audio pieces have the same timestamps with which video GOPs, then arrange the correct file saving operations by yourself.
Can you point me in the right direction? There seems to be a limited amount of information available. Are there any other resources except the one that I mentioned above?
