Rotate bounding box in DeepStream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hardware Platform: dGPU
**• DeepStream Version: 7.0 **
• TensorRT Version: 8.6.1
• NVIDIA GPU Driver Version: 550
• Issue Type: question

Does deepstream support rotated bounding box? My work flow is plate detect-> text detect( the red rotated bounding box in image) -> text recognition

image
image661×300 44.3 KB

3

I think deepstream doesn’t support rotated bounding box. You can use alignment in detected plate.

4

No. But we have the corresponding OCDR samples you can refer to. deepstream-nvocdr-app

5

After some research, i guess we have to implement this with nvdspreprocess. However, it seems that all the cropped object is resized with processing-height and processing-width. Is there any way to get cropped object without it being resized.

6

Or we can get the whole frame out and do cropping our-self but currently we have no clue, the API is a bit ambiguous. Please help us with 2 problem with nvdspreprocess:

  1. Can we get the cropped object without it being scaled.
  2. If not, can we get the frame data with frame-meta or batch-meta.
7

  • We don’t support draw the rotated bounding box currently. You need to get the coordinates and then draw it through nvdsosd yourself.

  • What do you intend to do with nvdspreprocess? Could you briefly introduce your model?

  • We already have similar samples. You can take a look to see if it meets your requirements. deepstream_lpr_app and deepstream-nvocdr-app

8

The problem is i have to do custom warping on input image and nvidia example doesnt provide it, so our plan is to use custom tensor with nvdspreprocess like this image -> text detection (4 points to make a polygon) -> nvdspreprocess (use source image, crop and warp using the polygon) -> input tensor -> text recognition.

9

We suggest that you first refer to the following sample deepstream-pose-classification. This sample covers how to handle the buffer and how to prepare the tensor data. And you need to implement the warp algorithm yourself.

10

I already go through it but is there anyway to get FULL FRAME with nvdspreprocess API? We will have to do the cropping ourself from the full frame since using NvBufSurfaceParams *surf_params = batch->units[i].roi_meta.converted_buffer; already scaled down my object.

11

Why do you need to obtain the FULL FRAME? According to your pipeline, shouldn’t you obtain the images of the detected objects from the text detection?

12

Yes i need cropped detected object but it is automatically scaled to network height and width in deepstream, we are hoping we can get the cropped object before it get scaled.

13

You can get the frame from the in_buf below.

sources\gst-plugins\gst-nvdspreprocess\gstnvdspreprocess.cpp
static GstFlowReturn
gst_nvdspreprocess_on_frame (GstNvDsPreProcess * nvdspreprocess, GstBuffer * inbuf,
    NvBufSurface * in_surf)
{
...
          if (scale_and_fill_data (nvdspreprocess, in_surf->surfaceList + batch_index,
                  &rect_params, scale_ratio_x, scale_ratio_y, offset_left, offset_top,
                  memory->surf, memory->surf->surfaceList + idx,
                  memory->frame_memory_ptrs[idx]) != GST_FLOW_OK) {
...
}
14

Thanks for helping, we will look forward to it <3.

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.