How to add the metadata to the H264 SEI data

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.3
• TensorRT Version 8.5.3-1+cuda11.8
• NVIDIA GPU Driver Version (valid for GPU only) 550.142
• Issue Type( questions, new requirements, bugs) questions

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
Python 3.8.10

I tried using the Python version of Deepstream to add custom SEI information to H264 encoding, but I was unable to succeed.
I used the following code, but couldn’t find any information in SEI.
This code from deepstream_python_apps/apps/deepstream-imagedata-multistream-redaction/deepstream_imagedata-multistream_redaction.py at master · NVIDIA-AI-IOT/deepstream_python_apps

def add_custom_metadata(pad, info):
    gst_buffer = info.get_buffer()
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    pyds.nvds_acquire_meta_lock(batch_meta)
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            frame_number = frame_meta.frame_num
        except StopIteration:
            break
        
        user_meta = pyds.nvds_acquire_user_meta_from_pool(batch_meta)
        if user_meta:
            test_string = 'test message ' + str(frame_number)
            print('adding user meta', test_string)
            data = pyds.alloc_custom_struct(user_meta)
            data.message = test_string
            data.message = pyds.get_string(data.message)
            data.structId = frame_number
            data.sampleInt = frame_number + 1

            user_meta.user_meta_data = data
            user_meta.base_meta.meta_type = pyds.NvDsMetaType.NVDS_USER_META

            pyds.nvds_add_user_meta_to_frame(frame_meta, user_meta)
        else:
            print('failed to acquire user meta')
            
        try:
            l_frame = l_frame.next
        except StopIteration:
            break

    pyds.nvds_release_meta_lock(batch_meta)
    return Gst.PadProbeReturn.OK

...
    streammux_src_pad = streammux.get_static_pad("src")
    streammux_src_pad.add_probe(Gst.PadProbeType.BUFFER, add_custom_metadata)
...
    nvdsmetainsert = Gst.ElementFactory.make("nvdsmetainsert", "meta-insert")
    nvdsmetainsert.set_property("serialize-lib", "libnvds_sei_serialization.so")
    pipeline.add(nvdsmetainsert)
...
    caps.link(nvdsmetainsert)
    nvdsmetainsert.link(encoder)
...

I have tried inserting metadata in different locations, but it seems to have had no effect

** Reference **
Gst-nvdsmetautils — DeepStream documentation

3

Have you tried our use-cases-sample-pipelines to check if the SEI information is added to the h264 encoding first?

4

I have tried it, and it works. I ran the following code and successfully received the SEI information on the client side:

gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=1280 height=720 batch-size=1 ! nvdsvideotemplate dummy-meta-insert=1 customlib-name=libcustom_videoimpl.so customlib-props="scale-factor:0.5" ! nvmultistreamtiler width=640 height=480 ! nvdsmetainsert serialize-lib="libnvds_sei_serialization.so" ! nvv4l2h264enc bitrate=1000000 ! h264parse ! flvmux streamable=true ! rtmpsink location=rtmp://push.stream_domain.cn/live/stream
5

Then you can refer to our source code first. If you want to customize the SEI information, please customize your code in the following C/C++ source code.

  1. add the NVDS_DUMMY_BBOX_META type user data in the videotemplate
sources\gst-plugins\gst-nvdsvideotemplate
void update_dummy_meta_data_on_buffer (NvDsBatchMeta *batch_meta)
{
  1. serialize the NVDS_DUMMY_BBOX_META type user data to the SEI NAL.
sources\gst-plugins\gst-nvdsmetautils\sei_serialization\sei_serialization.cpp
extern "C" void serialize_data (GstBuffer *buf);
void serialize_data (GstBuffer *buf)
{
6

Thank you for the help you provided. I have resolved the issue.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.