Identify a numpy frame fed into the pipeline

Hello, I’d like to follow up on a topic that has been closed: Match NvDsFrameMeta with their original gstreamer Buffer

I am sending multiple numpy images through a pipeline.

I use appsrc to feed numpy arrays to my pipeline:

img = cv2.imread("/opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.jpg")
img = cv2.resize(img, (1080, 1920))
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGBA)
buffer = Gst.Buffer.new_wrapped(img.tobytes())
self.source_bin_appsrc.emit("push-buffer", buffer)

The pipeline works fine, and I am able to retrieve the numpy image as well as the metadata from an object of type NvDsBatchMeta.

The problem is that once I get the NvDsBatchMeta I need to know what was the original numpy frame that this metadata refers to. In this case, I can’t simply download the image again from the pipeline.

The reason is the following: I am implementing a REST API. Suppose I have two users, user_A and user_B. They both send their images image_A and image_B to my API. My API sends the image to the pipeline. At the end of the pipeline I get the metadata for each image. However, how can I know what are the metadata that I need to send back to user_A, and what are the metadata for user_B?

I need to store some value into the image. Example: if I could attach to image_A the value user=user_A, then, at the end of the pipeline, when I get the metadata, I could do something like metadata.user and get the user that needs to receive the metadata.

In the post I linked above, @yuweiw said that: “You can consider using NvDsUserMeta to add your own tag for each gstbuffer through the probe function.”. However, how can I create a NvDsFrameMeta with attached NvDsUserMeta from a numpy array and send it to the pipeline?

Thanks

You can refer our python source code:

 user_event_meta = pyds.nvds_acquire_user_meta_from_pool

https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-test4/deepstream_test_4.py

Hey @yuweiw , thank you for your reply.
I think that code would be useful to add metadata.
However, in my case, I have a numpy array. And before adding metadata, I need to create a NvDsFrameMeta from that numpy array. How can I do that?
Thank you

Ok, I got what you mean. This binding demo is in our roadmap. Your needs are similar to this:
https://forums.developer.nvidia.com/t/attaching-user-metadata-to-gstreamer-buffer-using-python-bindings/223486
We are working on it. Thanks

Hi @yuweiw ,

You are correct. My needs are similar to the ones mentioned in the link.
It’s great to know you are working on it. Well done!

Thank you