Adding metadata (frame information) into frame streamed by WebRTC

Hi,
I’m building an AI server that runs on an NX Orin. It’s running OK and I used jetson-inference: jetson-inference/docs/aux-streaming.md at master · dusty-nv/jetson-inference · GitHub
to stream video output into the browser.

However, now I want to send the metadata (detection/tracking information) along with the frame to the browser. Is there any way that I can add this information(like a JSON) into the frame and read it on the browser side?

Hi @manh.ntm3, I believe that WebRTC has the concept of ‘data channels’ that could theoretically be used to transport such metadata, however it is complex to integrate and the server-side WebRTC implementation that I use through GStreamer does not implement such protocols. Hence you would either need to transmit your metadata out-of-band to the client (i.e. via websockets) or use an alternate solution. DeepStream has a much more complete stack available for working with metadata for analytics via various message brokers like Kafka, Redis, ect.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.