# Get image gst-buffer created by from URIDECODEBIN

**URL:** <https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762>\
**Category:** DeepStream SDK\
**Created:** [May 3, 2024, 1:10pm UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762 "2024-05-03T13:10:49Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 3, 2024, 1:10pm UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/1 "2024-05-03T13:10:49Z")

</div>

**• Hardware Platform (Jetson / GPU)** :- GPU  
**• DeepStream Version** :- 6.4  
**• TensorRT Version**  
**• NVIDIA GPU Driver Version (valid for GPU only)** :- 545  
**• Issue Type( questions, new requirements, bugs)** :- Question

I have a deepstream application that is using URIDECODEBIN as the source element. The pipeline looks like  
`uridecodebin -> nvvideoconvert -> streammux -> rest-of-the-pipeline`

the uridecodebin element is reading an RTSP stream. I want the pipeline to work only on some part of the video so, I’m using `src-crop` property of `nvvideoconvert` element to crop the incoming RTSP stream and feed only the required ROI to the pipeline. But because of this I loose the full FOV image coming from the camera.

**My Question is** :- is there any way to do that ? or whats the best way to do that ?

**What I have tried so far** :-

- Trial One  
I thought of attaching a probe function at `sink pad` of `nvvideoconvert` and access the full fov image and attach it as a custom gst-metadata to the buffer. but the linking of `uridecodebin -> jpegenc -> nvvideoconvert` creates some linking issues (probably related to caps). Im uploading  
[error\_logs.txt](https://forums.developer.nvidia.com/uploads/short-url/6HYFQJ5jF2Z7C4KQX6R0i5Dpidv.txt) (695 Bytes)  
if you’d want to check that.

- Trial Two  
I create two queues. One containing Full FOV images another containing the cropped ones. Cropped ones are feeded to the pipeline, after inference is over, I try to map the inference results from the cropped images to the full FOV images. I dont like this approach as The mapping is often not right.

**More information if it helps** :- The elements created inside/by uridecodebin for the type of stream that Im using are (in order) :- `depay-> parser-> tee_rtsp_pre_decode-> dec_que-> tee_rtsp_post_decode-> decodebin-> queue-> nvvidconv-> src_cap_filter_nvvidconv-> h264parse0-> capsfilter0-> nvv4l2decoder0`

---

<div class="post-metadata">

**Author:** ![zgjja](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@zgjja](https://forums.developer.nvidia.com/u/zgjja)\
**Post date:** [May 3, 2024, 6:29pm UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/2 "2024-05-03T18:29:31Z")

</div>

I think It depends on your task, if you are just using `nvinfer` or `nvinferserver`, you can write your own postprocess logic to filter results that are out of your ROI, but if your pipeline does some operation on the original image, this may not work as you expected.

Your solution 1 is not good but 2 seems better (but still can’t meet your requirement). Anyway i prefer do postprocess inside `nvinferserver` or `nvinfer`

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 4, 2024, 9:53am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/3 "2024-05-04T09:53:43Z")

</div>

You don’t need to use `nvvideoconvert` to crop the video. I suggest you use `nvdspreprocess`. You only need to specify the ROI.

Please refer to the document below

[https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS\_plugin\_gst-nvdspreprocess.html#id4](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvdspreprocess.html#id4)

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 9, 2024, 5:39am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/5 "2024-05-09T05:39:18Z")

</div>

cant use `nvdspreprocess` because we have multiple sources and the cropping needs to happen before the streammux.

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 9, 2024, 7:24am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/7 "2024-05-09T07:24:33Z")

</div>

> [@user156910](#):
>
> have multiple sources and the cropping needs to happen before the streammux.

nvdspreprocess can support multiple sources. Just set `roi-params-src-<id>` in nvdspreprocess configuration file for every source.

Is cropping necessary?

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 9, 2024, 7:44am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/8 "2024-05-09T07:44:19Z")

</div>

yes cropping is necessary. I want the full resolution (whatever the camera supports) image, to be given to the pipeline for inference. so we take the full resolution image from camera, crop the required ROI and then feed it to streammux.

if we feed the image coming from camera without cropping, in most of the cases, camera is capturing regions where we dont really have any object.

Also, we have to run the pipeline on different cameras having different resolutions, if we dont crop the ROI, the streammux will rescale all of them to one resoution, which may result in out objects getting very small in size.

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 9, 2024, 11:13am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/9 "2024-05-09T11:13:19Z")

</div>

Do you need a pipeline similar to the one below?

```auto
                     |-----------> FOV image --> sink
uridecodebin --> tee |
                     |-----------> nvvideoconvert (do crop) --> nvstreammux --> infer --> osd --> sink

```

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 9, 2024, 11:52am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/10 "2024-05-09T11:52:28Z")

</div>

yes.

```auto
                     |-----------> FOV image --> sink
uridecodebin --> tee |
                     |-----------> nvvideoconvert (do crop) --> nvstreammux --> infer --> osd --> sink

```

the task is to obtain the full image before we crop it. if it can be done without splitting the pipeline we are okay.

so maybe,

```auto
                     
uridecodebin ---------> nvvideoconvert (do crop) --> nvstreammux --> infer --> osd --> sink\
                   |
                   | 
       we can also put a probe here,
        to obtain the full image before we crop it.  

```

once we obtain the full image, we plan on attaching it as custom get-meta to the gst-buffers.

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 9, 2024, 11:55am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/11 "2024-05-09T11:55:17Z")

</div>

why we were thinking of spllitting the pipeline in the first place is because we thought in the second head of the pipeplien we would put `nvjpegenc` or somehing that will encode the iamge from raw data to jpeg. But thats okay, splitting the pipeline is not something that is compelled upon us.

what matters is, getting the full frame before it gets cropped. and attaching it as custom gst meta.

we are open to do it in any way that works.

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 10, 2024, 2:10am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/12 "2024-05-10T02:10:18Z")

</div>

> [@user156910](#):
>
> once we obtain the full image, we plan on attaching it as custom get-meta to the gst-buffers.

If the requirement is to save images in the probe function, you can refer to `deepstream-image-meta-test`. `nvds_obj_enc_process` can help you complete this work very well.

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 10, 2024, 8:51am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/13 "2024-05-10T08:51:31Z")

</div>

we already have that probe in our pipeline.  
that WONT work here as i want to get the full frame BEFORE streammux.

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 10, 2024, 8:52am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/14 "2024-05-10T08:52:25Z")

</div>

> [@junshengy](#):
>
> `nvds_obj_enc_process`

can you please read the question, understand it and then reply?

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 10, 2024, 9:12am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/15 "2024-05-10T09:12:38Z")

</div>

Do you understand what I said?

> [@user156910](#):
>
> ```auto
> uridecodebin ---------> nvvideoconvert (do crop) --> nvstreammux --> infer --> osd --> sink\
> |
> | 
> we can also put a probe here,
> to obtain the full image before we crop it.  
> 
> ```

If this is the pipeline you need, add a probe function in the src pad of `uridecodebin`, `nvds_obj_enc_process` can save the full frame as an image and add a custom gst-meta (NVDS\_CROP\_IMAGE\_META). Cropping and inference will also work correctly

If not, please share your pipeline

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 10, 2024, 9:34am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/16 "2024-05-10T09:34:18Z")

</div>

```static
pgie_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer ctx)
{
  GstBuffer *buf = (GstBuffer *) info->data;
  GstMapInfo inmap = GST_MAP_INFO_INIT;
  if (!gst_buffer_map (buf, &inmap, GST_MAP_READ)) {
    GST_ERROR ("input buffer mapinfo failed");
    return GST_PAD_PROBE_DROP;
  }
  NvBufSurface *ip_surf = (NvBufSurface *) inmap.data;
  gst_buffer_unmap (buf, &inmap);

  NvDsObjectMeta *obj_meta = NULL;
  guint vehicle_count = 0;
  guint person_count = 0;
  NvDsMetaList *l_frame = NULL;
  NvDsMetaList *l_obj = NULL;
  NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
  const gchar *calc_enc_str = g_getenv ("CALCULATE_ENCODE_TIME");
  gboolean calc_enc = !g_strcmp0 (calc_enc_str, "yes");

  for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
    NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);

```

this is a part of the probe function that you are talking about. My point is, if I put it before streammux, it will not work as there is no NVDS meta. am I right ??

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 10, 2024, 10:05am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/17 "2024-05-10T10:05:30Z")

</div>

Ok, I got your point, you can modify the pipeline to look like the following, which is similar to this [GitHub - NVIDIA-AI-IOT/deepstream\_parallel\_inference\_app: A project demonstrating how to use nvmetamux to run multiple models in parallel.](https://github.com/NVIDIA-AI-IOT/deepstream_parallel_inference_app?tab=readme-ov-file).

I think it should work.

```auto
uridecodebin --> nvstreammux ----> nvstreamdemux --> nvviddeoconvert (do crop) --> nvstreammux --> .....
							|
							|
                       probe at here

```

You can also add custom gst-meta using `gst_buffer_add_meta`. This does not depend on `nvstreammux` and is not compatible with `NvDsMetaType`. Of course `nvds_obj_enc_process` will not work either. You can use software encoding if you want.

[https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c#gst\_buffer\_add\_meta](https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c#gst_buffer_add_meta)

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 10, 2024, 10:31am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/18 "2024-05-10T10:31:51Z")

</div>

```auto
> uridecodebin --> nvstreammux(first) ----> nvstreamdemux --> 
---> nvviddeoconvert (do crop) --> nvstreammux(second) --> .....
> |
> |
> probe at here 

```

this seems interesting. but since I have different cameras running at different resolution, I would still have to give some width and height to streammux(one).

Im thinking of creating a bin, an Input bin containing `uridecodebin->streammux->nvstreamdemux->nvvideoconvert`

each source will have this unique bin and streammux in this bin will get the stream resolution so it doesnt rescale the image.

I think that should work. what do you think?

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [May 10, 2024, 10:47am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/19 "2024-05-10T10:47:43Z")

</div>

You can try it. If you need the rtsp reconnection feature, you can consider using `nvurisrcbin` instead of `uridecodebin`.

---

<div class="post-metadata">

**Author:** ![user156910](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user156910](https://forums.developer.nvidia.com/u/user156910)\
**Post date:** [May 10, 2024, 11:17am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/20 "2024-05-10T11:17:27Z")

</div>

> [@junshengy](#):
>
> `nvurisrcbin`

yes. thanks.

---

<div class="post-metadata">

**Author:** ![yingliu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/yingliu/32/136703_2.png) [@yingliu](https://forums.developer.nvidia.com/u/yingliu)\
**Post date:** [June 4, 2024, 5:57am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/21 "2024-06-04T05:57:44Z")

</div>

**There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks**

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [June 18, 2024, 5:58am UTC](https://forums.developer.nvidia.com/t/get-image-gst-buffer-created-by-from-uridecodebin/291762/22 "2024-06-18T05:58:04Z")

</div>

Closed on June 18, 2024, 5:58am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
