Why I use NvBufSurfTransformCompositeBlend function got VIC operation is not supported error?

Hi, I am using NvBufSurfTransformCompositeBlend function, but got the error: "VIC operation is not supported
". So I want to verify is the function NvBufSurfTransformCompositeBlend not supported on VIC ? or other reasons.
https://docs.nvidia.com/jetson/archives/r35.4.1/ApiReference/group__ds__bbb.html#ga30737c04cf7ebf543ec4c10f9cb3b3ff

Now my NvBufSurface type is:
src0 : NVBUF_COLOR_FORMAT_RGBA
src1 : NVBUF_COLOR_FORMAT_RGBA
alpha: NVBUF_COLOR_FORMAT_GRAY8

My jetpack is 5.1.2. Thanks for your help, and my codes are shown as below:

void CameraStitcher::CompositeImage() {
   // get NvBufSurface from cv::Mat
   NvBufSurface** composite_imput = new NvBufSurface*[2];
   for (int i = 0; i < 2; i++) {
      NvBufSurfaceCreateParams create_params;
      create_params.gpuId = 0;
      create_params.width = crop_info_.width;
      create_params.height = crop_info_.height;
      create_params.size = 0;
      create_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
      create_params.layout = NVBUF_LAYOUT_PITCH;
      create_params.memType = NVBUF_MEM_DEFAULT;

      if (NvBufSurfaceCreate(&composite_imput[i], 1, &create_params) != 0) {
         std::cout<<"Nv surface create error"<<std::endl;
      }
      composite_imput[i]->numFilled = 1;

      if (NvBufSurfaceMap(composite_imput[i], 0, 0, NVBUF_MAP_READ_WRITE) != 0){
         std::cout<<"Nv surface map error"<<std::endl;
      }

      cv::Mat remap_image = cv::imread("test_" + std::to_string(i) + ".png");
      cv::cvtColor(remap_image, remap_image, cv::COLOR_BGR2RGBA);
      memcpy(composite_imput[i]->surfaceList[0].mappedAddr.addr[0], remap_image.ptr(),
         remap_image.total() * remap_image.elemSize());
   }

   // get alpha from CV_8UC1 cv::Mat
   NvBufSurface* alpha; 
   NvBufSurfaceCreateParams mask_params;
   mask_params.gpuId = 0;
   mask_params.width = crop_info_.width;
   mask_params.height = crop_info_.height;
   mask_params.size = 0;
   mask_params.colorFormat = NVBUF_COLOR_FORMAT_GRAY8;
   mask_params.layout = NVBUF_LAYOUT_PITCH;
   mask_params.memType = NVBUF_MEM_DEFAULT;

   if (NvBufSurfaceCreate(&alpha, 1, &mask_params) != 0) {
      std::cout<<"Nv surface create error"<<std::endl;
   }
   alpha->numFilled = 1;

   if (NvBufSurfaceMap(alpha, 0, 0, NVBUF_MAP_READ_WRITE) != 0){
      std::cout<<"Nv surface map error"<<std::endl;
   }
   cv::Mat mask = camera_masks_[0].clone();  // CV_8UC3 type
   memcpy(alpha->surfaceList[0].mappedAddr.addr[0], mask.ptr(),
      mask.total() * mask.elemSize());

   // initialize output
   NvBufSurf::NvCommonAllocateParams input_params = {0};
   input_params.width = crop_info_.width;
   input_params.height = crop_info_.height;
   input_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
   input_params.layout = NVBUF_LAYOUT_PITCH;
   input_params.memType = NVBUF_MEM_DEFAULT;

   int composited_frame = 0;
   if(-1 == NvBufSurf::NvAllocate(&input_params, 1, &composited_frame)) {
      std::cout<<"NvAllocate error"<<std::endl;
   }
   if (!composited_frame) {
      std::cout<<"Failed to allocate composited buffer"<<std::endl;
   }

   NvBufSurface *output;
   if (-1 == NvBufSurfaceFromFd(composited_frame, (void**)(&output))) {
      std::cout<<"Failed to get output"<<std::endl;
   }

   // get params
   NvBufSurfTransformCompositeBlendParams composite_param;
   memset(&composite_param, 0, sizeof(composite_param));
   composite_param.composite_blend_flag = NVBUFSURF_TRANSFORM_BLEND;
   composite_param.input_buf_count = 2;
   composite_param.composite_blend_filter = NvBufSurfTransformInter_Bilinear;

   int comp_value =NvBufSurfTransformCompositeBlend(composite_imput[0], composite_imput[1], alpha, output, &composite_param);
   std::cout<<"comp_value "<<comp_value<<std::endl;
   return;
}

Hi,
Please share your use-case so that we can check and suggest next. The error you hit is because this NvBufSurface has different format:

alpha: NVBUF_COLOR_FORMAT_GRAY8

Hi DaneLLL, thanks for your reply.
My use-case is that I want to stitch two images A and B, and get a new image C. The blending weights for each pixel in image A are stored in matrix D, which is alpha. Matrix A/B/C/D have same width and height. The value for output matrix C is calculated by:
C_value = (A_value * weight + B_value * (255-alpha)) / 255

In my code, matrix A and B are stored in “composite_imput”, matrix D is stored in “alpha”.

Hi DaneLLL, I tried to use NVBUF_COLOR_FORMAT_RGBA for alpha, but got the same error: VIC operation is not supported.
The way I construct alpha is that:

 // get alpha from cv::Mat
 NvBufSurface* alpha; 
 NvBufSurfaceCreateParams mask_params;
 mask_params.gpuId = 0;
 mask_params.width = width;
 mask_params.height = height;
 mask_params.size = 0;
 mask_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
 mask_params.layout = NVBUF_LAYOUT_PITCH;
 mask_params.memType = NVBUF_MEM_DEFAULT;

 if (NvBufSurfaceCreate(&alpha, 1, &mask_params) != 0) {
    std::cout<<"Nv surface create error"<<std::endl;
 }
 alpha->numFilled = 1;

 if (NvBufSurfaceMap(alpha, 0, 0, NVBUF_MAP_READ_WRITE) != 0){
    std::cout<<"Nv surface map error"<<std::endl;
 }

 cv::Mat mask1 = camera_masks_[0].clone();  // CV_8UC1 type
 cv::Mat mask2 = camera_masks_[0].clone();  // CV_8UC1 type
 cv::Mat mask3 = camera_masks_[0].clone();  // CV_8UC1 type
 std::vector<cv::Mat> camera_mask_array{mask1, mask2, mask3};
 cv::Mat mask;
 cv::merge(camera_mask_array, mask);           // CV_8UC3 type
 cv::cvtColor(mask, mask, cv::COLOR_BGR2RGBA); 
 memcpy(alpha->surfaceList[0].mappedAddr.addr[0], mask.ptr(),
    mask.total() * mask.elemSize());

Hi,
The function NvBufSurfTransformCompositeBlend() does not support applying different alpha value to each pixel. You would need to implement CUDA code for this step. A possible solution is:

  1. Call NvBufSurfTransformCompositeBlend() to stitch A and B NvBufsurface into C NvBufsurface
  2. Call NvBufsurface and CUDA APIs to get CUDA pointer of C NvBufsurface
  3. Implement CUDA code to apply alpha value to R, B, B channels of C NvBufsurface

Here is a patch for using NvBuffer APIs:
Resources for custom nvivafilter for Jetpack 5.1.1 Jetson NX - #11 by DaneLLL

NvBuffer APIs are replaced with NvBufsurface APIs on Jetpack 5. Although the patch is based on NvBuffer, you can find similar APIs in NvBufsurface.

NvBufSurfTransformCompositeBlend() is demonstrated in 13_argus_multi_camera sample.

Hi, Thanks for your reply!
I really want to use VIC to do this thing, because the limited computing power of GPU and CPU.
I want to know if thera a method for matrix multiplication with NvBufSurf on VIC device, like A.mul(B) method in OpenCV. If yes, I can also implement this function on VIC.

Hi,
VIC does not support the function. Please use the OpenCV API, or implement CUDA code.

Got it! Thank you very much!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.