Gstreamer metadata output recommendations

I’m loving DeepStream 4 so far. It does more than I expected, and the examples are fantastic, but I do have a few questions. I’m hoping to put the per-frame detection metadata inside my output video file if possible. So far I’m thinking of using mkvmux’s subtitle track to accomplish the task, but I figured i’d ask if anybody knows of a container that supports this sort of generic per-frame metadata (tagged, specifically as metadata) or if Nvidia has any recommendations on this. Is a sidecar preferable? it seems easier to implement.

Hi,
mkv looks to be a good choice. Other users may also share their knowledge.

Using track atoms in mov should also work.
https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html