Synchronizing Frame Metadata with MP4 Output from Smart Record

[junshengy]

@curious_cat

What is the best way to ensure metadata and video are aligned frame-accurately

@curious_cat

• Would embedding metadata (e.g., via nvdsmeta) directly into the MP4 help?

Are you planning to store metadata in the mp4 container?
You can convert metadata to h264/5 sei, the store SEI to mp4 container. But this requires you to parse the mp4 file and extract the SEI.

Can you share your goals for doing so?

Refer to this topic

Gstreamer Pipeline to insert metadata using DeepStream

This function cannot be fully implemented through Python, which requires modifying native bindings. Please refer to the patch below. Then recompile and install pyds. diff --git a/apps/deepstream-custom-binding-test/deepstream_custom_binding_test.py b/apps/deepstream-custom-binding-test/deepstream_custom_binding_test.py index 5e41846c..c4d8b28a 100644 --- a/apps/deepstream-custom-binding-test/deepstream_custom_binding_test.py +++ b/apps/deepstream-custom-binding-test/deepstream_custom_binding_t…

If your application is written by native code. Please refer to /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdsmetautils/sei_serialization .

@curious_cat

Or should the metadata be stored externally as a sidecar file (e.g., Avro/JSON), indexed by frame-level PTS?

I think this is feasible. Each json file has the PTS and source id of the corresponding frame.