Issue with plotting on image in deepstream-app

ok thanks let me try. Now no compile errors.

Nothing is displayed. What could be wrong with code.

NvBufSurface* roi_surface;
  NvBufSurface* color_surface;
  NvBufSurface* alpha_surface;
  NvBufSurfaceCreateParams create_params;
  create_params.gpuId  = dsexample->gpu_id;  
  create_params.size = 0;
  create_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
  create_params.layout = NVBUF_LAYOUT_PITCH;
  create_params.memType = NVBUF_MEM_DEFAULT;
  create_params.memType = NVBUF_MEM_CUDA_UNIFIED;
  int x1 = refboxes_display[source].corners[boxidx+0];  int y1 = refboxes_display[source].corners[boxidx+1];
  int x2 = refboxes_display[source].corners[boxidx+2];  int y2 = refboxes_display[source].corners[boxidx+3];
  create_params.width  = x2 - x1;
  create_params.height = y2 - y1;
  cv::Mat roi = in_mat(cv::Rect(x1, y1, x2-x1, y2-y1 )); 
  cv::Mat color(roi.size(), CV_8UC4, cv::Scalar( 0, 0, 255 ));
  cv::Mat alpha(roi.size(), CV_8UC4, cv::Scalar( 0, 0, 0 ));
  if (NvBufSurfaceCreate (&roi_surface, 1, &create_params) != 0) {
	 printf ("Error: in Create\n");
 }
  NvBufSurfaceMemSet (roi_surface, 0, 0, 0);
  if (NvBufSurfaceMap (roi_surface, 0, 0, NVBUF_MAP_READ_WRITE) != 0){
	printf("Err in Map\n");
  }
  if (NvBufSurfaceSyncForCpu (roi_surface, 0, 0) !=0) {
			    printf("Err in Synccpu");
  }
  memcpy(roi_surface->surfaceList[0].mappedAddr.addr[0], roi.ptr(), create_params.width * create_params.height * 4);
   if (NvBufSurfaceCreate (&color_surface, 1, &create_params) != 0) {
	printf ("Error: in Create\n");
  }
   NvBufSurfaceMemSet (color_surface, 0, 0, 0);
   if (NvBufSurfaceMap (color_surface, 0, 0, NVBUF_MAP_READ_WRITE) != 0){
	 printf("Err in Map\n");
   }
   if (NvBufSurfaceSyncForCpu (color_surface, 0, 0) !=0) {
	 printf("Err in Synccpu");
  }
  memcpy(color_surface->surfaceList[0].mappedAddr.addr[0], color.ptr(), create_params.width * create_params.height * 4);
  if (NvBufSurfaceCreate (&alpha_surface, 1, &create_params) != 0) {
	printf ("Error: in Create\n");
  }
   NvBufSurfaceMemSet (alpha_surface, 0, 0, 0);
   if (NvBufSurfaceMap (alpha_surface, 0, 0, NVBUF_MAP_READ_WRITE) != 0){
        printf("Err in Map\n");
   }
 if (NvBufSurfaceSyncForCpu (alpha_surface, 0, 0) !=0) {
       printf("Err in Synccpu");
 }
memcpy(alpha_surface->surfaceList[0].mappedAddr.addr[0], alpha.ptr(), create_params.width * create_params.height * 4);
_NvBufSurfTransformCompositeBlendParams *params;
 NvBufSurfTransformCompositeBlend(roi_surface, color_surface, alpha_surface, roi_surface, params);

Is it possible to share your whole sample, including the Makefile ?

Thanks!

gst-dsexample.zip (117.0 KB)

May I know any updates? Why nothing is displayed though no errors in compile and runtime?

Hello may I know any updates on plotting mask on the image? Thank you

Hi @edit_or ,
setting up the repo, will check and reply. sorry for delay!

thanks Sir.

Hi @edit_or
Sorry! seems NvBufSurfTransformCompositeBlend() is an very old API, it may not work.

Back to the original question, I think , you want to draw rectangles on the frames, right? Is the rectangles the BBOX from inference models?

As you may know, below is the command DS pipeline, the inference plugin will generate some BBOX and attach to DS metadata, then OSD plugin will extra the BBOX metadata and draw on the frames. Can this meet your requirmeent?

multiple streams --> decoding (nvv4l2decoder) ---> batching (nvstreammux) --> inference (nvinfer / nvinferserver) ---> OSD (nvdsosd) ---> tiler --> encoding / render

No. I like to draw my own mask showing region of interest on image. How can I draw?

Memory leak of opencv in deepstream can’t be solved?

Hi I am going to add mask params and according to _NvOSD_MaskParams Struct Reference.

float * data inside Struct represents the whole mask area in image or just contour of mask?

_NvOSD_MaskParams a;
object_meta->mask_params.push_back(a);

If push back custom maskparam to object_meta->mask_params, NvOSD can plot mask on image, right?

Hi @edit_or ,
Really sorry for long delay!

Regarding the mask, please check deepstream_tao_apps/nvdsinfer_custombboxparser_tao.cpp at release/tao3.0 · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub , and you can run below command to check its output

$ SHOW_MASK=1 ./apps/tao_detection/ds-tao-detection -c configs/peopleSegNet_tao/pgie_peopleSegNetv2_tao_config.txt -i /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264