# VPI image pointing to managed memory

**URL:** <https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775>\
**Category:** Jetson AGX Xavier\
**Tags:** vpi\
**Created:** [September 27, 2023, 12:18pm UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775 "2023-09-27T12:18:42Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [September 27, 2023, 12:18pm UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/1 "2023-09-27T12:18:42Z")

</div>

I have a memory buffer allocated with cudaMallocManaged; this buffer is used as input to the AI model as a binding. I do the image preprocessing with the OpenCV as follows: I create and OpenCV matrix pointing to that buffer on GPU and preprocess images inplace. Like:

cv::Size inputSize = \_modelConfig.inputSize;  
int numBindings = engine-\>getNbIOTensors();  
void \*buffer\_in, \*buffer\_out; // buffers for input and output of the model  
int size\_in = inputSize.height \* inputSize.width \* 3 \* sizeof(float);  
cudaMallocManaged(&buffer\_in, size\_in);  
int size\_out = 1 \* sizeof(float); // output is juts a float number  
cudaMallocManaged(&buffer\_out, size\_out);

void\*\* bindings = new void\*[numBindings];  
bindings[0] = buffer\_in;  
bindings[1] = buffer\_out;

// create cv::cuda::GpuMat outputMat pointing to bindings[0] what is buffer\_in  
cv::cuda::GpuMat outputMat = cv::cuda::GpuMat(inputSize.height, inputSize.width, CV\_32FC3, bindings[0]);  
preprocess outputMat with the OpenCV…

Now I want to do the same with the VPI library. But it cannot point to the Managed buffer bindings[0]. It requires whether NvBufSurface or cudaArray. Is it possible to make for example, NvBufSurface from the managed memory? What would you advice in order to make a VPIImage pointing to the bounding[0]?

With the best wishes,  
Valeriy

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [September 28, 2023, 3:35am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/3 "2023-09-28T03:35:40Z")

</div>

Hi,

VPI can wrap the GPU buffer.  
Which VPI version do you use? 1.2 or 2.x?

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [September 28, 2023, 6:23am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/4 "2023-09-28T06:23:36Z")

</div>

I use vpi 2.  
How can I wrap the GPU buffer? As I see it, the code should look like:

cudaMallocManaged(&GPU\_buffer, buffer\_size);  
VPIIMage img\_vpi;  
VPIImageData img\_data;  
img\_data.bufferType = ?;  
img\_data.buffer.? = GPU\_buffer;

VPIStatus err\_vpi = vpiImageCreateWrapper(&img\_data, nullptr, VPI\_BACKEND\_CUDA, &img\_vpi);

what should I use instead of “?”  
for buffertype there are options: VPI\_IMAGE\_BUFFER\_CUDA\_PITCH\_LINEAR, VPI\_IMAGE\_BUFFER\_CUDA\_ARRAY, VPI\_IMAGE\_BUFFER\_EGLIMAGE, VPI\_IMAGE\_BUFFER\_NVBUFFER

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [October 11, 2023, 11:37am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/5 "2023-10-11T11:37:59Z")

</div>

Could you please help me with my question?

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [October 12, 2023, 5:21am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/6 "2023-10-12T05:21:56Z")

</div>

Hi,

Sorry for the late update.  
Here is an example to wrap CUDA buffer with VPI 2.x for your reference:

```auto
// Given width, height, d_ptr(GPU buffer pointer)

VPIImage img;
VPIImageData data = {};
data.bufferType = VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR;
data.buffer.pitch.numPlanes = 1;
data.buffer.pitch.planes[0].width = width;
data.buffer.pitch.planes[0].height = height;
data.buffer.pitch.planes[0].data = d_ptr;

CHECK_STATUS(vpiImageCreateWrapper(&data, nullptr, 0, &img));

```

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [October 12, 2023, 11:58am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/7 "2023-10-12T11:58:08Z")

</div>

Thank you very much. I tried your code on Nvidia DevKit with Jetson AGX Xavier as follows:

void\* buff;  
int width = 100;  
int height = 100;  
cudaMalloc(&buff, width \* height \* 4 \* sizeof(float));  
VPIImage img\_vpi;  
VPIImageData img\_data = {};  
img\_data.bufferType = VPI\_IMAGE\_BUFFER\_CUDA\_PITCH\_LINEAR;  
img\_data.buffer.pitch.numPlanes = 1;  
img\_data.buffer.pitch.planes[0].width = width;  
img\_data.buffer.pitch.planes[0].height = height;  
img\_data.buffer.pitch.planes[0].data = buff;  
VPIStatus err\_vpi = vpiImageCreateWrapper(&img\_data, nullptr, 0, &img\_vpi);  
std::cout \<\< vpiStatusGetName(err\_vpi) \<\< std::endl;

the output is:  
VPI\_ERROR\_INVALID\_ARGUMENT

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [October 16, 2023, 6:26am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/8 "2023-10-16T06:26:57Z")

</div>

Hi,

We want to check the invalid argument error in our environment.  
Could you wrap the code into a runnable source and share it with us?

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [October 16, 2023, 9:26am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/9 "2023-10-16T09:26:36Z")

</div>

#include   
#include \<opencv2/cudaimgproc.hpp\>  
#include \<opencv2/imgcodecs.hpp\>  
#include \<opencv2/cudawarping.hpp\> // for cv::cuda::resize  
#include \<cuda.h\>  
#include \<cuda\_runtime.h\>  
#include \<vpi/Image.h\>  
#include \<vpi/Status.h\>  
#include \<vpi/ImageFormat.h\>

int main()  
{  
cuInit(0);  
int img\_width\_output = 224;  
int img\_height\_output = 224;  
int size\_output = img\_width\_output \* img\_height\_output \* 3 \* sizeof(float);  
void\* output\_buff; // a preprocessed image will be put here; this buffer will be used as a model input (binding)  
cudaMalloc(&output\_buff, size\_output);

```
// how I do it with OpenCV GPU version
cv::Mat img_input;
cv::cuda::GpuMat img_gpu_in, img_gpu_processed;
std::string path = "304194-17086108.jpg"; // here use your image
img_input = cv::imread(path);
img_gpu_in = cv::cuda::GpuMat(img_input); // build OpenCV GPU image based on the CPU version
// wrap an OpenCV GPU image around previously allocated output_buff
img_gpu_processed = cv::cuda::GpuMat(img_height_output, img_width_output, CV_32FC3, output_buff);
cv::cuda::resize(img_gpu_in, img_gpu_processed, cv::Size(img_height_output, img_width_output), 0, 0, cv::INTER_NEAREST);
// the resulting resized image is stored in output_buff and I can use it as model input in future

// now I try to do the same with VPIimage instead of OpenCV GPU version
VPIImage img_vpi;
VPIImageData img_data = {};
img_data.bufferType = VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR;
img_data.buffer.pitch.numPlanes = 1;
img_data.buffer.pitch.planes[0].width = img_width_output;
img_data.buffer.pitch.planes[0].height = img_height_output;
img_data.buffer.pitch.planes[0].pixelType = VPI_PIXEL_TYPE_3F32;
img_data.buffer.pitch.planes[0].data = output_buff;
//VPIStatus err_vpi = vpiImageCreateWrapper(&img_data, nullptr, VPI_BACKEND_CUDA, &img_vpi);
VPIStatus err_vpi = vpiImageCreateWrapper(&img_data, nullptr, 0, &img_vpi);
std::cout << vpiStatusGetName(err_vpi) << std::endl; // Output: VPI_ERROR_INVALID_ARGUMENT

img_gpu_in.release();
img_gpu_processed.release();
cudaFree(output_buff);

return 0;

```

}

Compile it with:  
nvcc -std=c++17 -I /usr/local/cuda/include/ -I /usr/local/include/opencv4/ -I /opt/nvidia/vpi2/include/ -lcuda -lcudart -lopencv\_core -lopencv\_imgcodecs -lopencv\_cudawarping -lnvvpi ForGuysFromNvidia.cu -o ForGuysFromNvidia

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [October 17, 2023, 5:50am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/10 "2023-10-17T05:50:26Z")

</div>

Thanks.

Will get back to you after checking.

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [October 26, 2023, 8:47am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/11 "2023-10-26T08:47:40Z")

</div>

Hi,

Below line is not correct:

```auto
img_data.buffer.pitch.planes[0].pixelType = VPI_PIXEL_TYPE_3F32;

```

Only below is supported in VPI Image:  
[https://docs.nvidia.com/vpi/ImageFormat\_8h.html](https://docs.nvidia.com/vpi/ImageFormat_8h.html)  
For example, please try this instead:

```auto
img_data.buffer.pitch.format = VPI_IMAGE_FORMAT_F32;

```

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [October 26, 2023, 9:43am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/12 "2023-10-26T09:43:25Z")

</div>

I replaced the line ```  
img\_data.buffer.pitch.planes[0].pixelType = VPI\_PIXEL\_TYPE\_3F32;

````auto
with ```
img_data.buffer.pitch.format = VPI_IMAGE_FORMAT_F32;

````

but still have the same output VPI\_ERROR\_INVALID\_ARGUMENT

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [November 2, 2023, 5:57am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/13 "2023-11-02T05:57:00Z")

</div>

Hi,

Could you try if the below setting can work?

```auto
VPIImageData data;
data.bufferType = VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR;
data.buffer.pitch.format = VPI_IMAGE_FORMAT_Y8_ER;
data.buffer.pitch.planes[0].data = image_ptr;
data.buffer.pitch.planes[0].width = image_width;
data.buffer.pitch.planes[0].height = image_height;
data.buffer.pitch.planes[0].pixelType = VPI_PIXEL_TYPE_DEFAULT;

```

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [November 2, 2023, 9:23am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/14 "2023-11-02T09:23:27Z")

</div>

I tried. The whole code is the following:

#include   
#include \<opencv2/cudaimgproc.hpp\>  
#include \<opencv2/imgcodecs.hpp\>  
#include \<opencv2/cudawarping.hpp\> // for cv::cuda::resize  
#include \<cuda.h\>  
#include \<cuda\_runtime.h\>  
#include \<vpi/Image.h\>  
#include \<vpi/Status.h\>  
#include \<vpi/ImageFormat.h\>

int main()  
{  
cuInit(0);  
int img\_width\_output = 224;  
int img\_height\_output = 224;  
int size\_output = img\_width\_output \* img\_height\_output \* 3 \* sizeof(float);  
void\* output\_buff; // a preprocessed image will be put here; this buffer will be used as a model input (binding)  
cudaMalloc(&output\_buff, size\_output);

```
// how I do it with OpenCV GPU version
cv::Mat img_input;
cv::cuda::GpuMat img_gpu_in, img_gpu_processed;
std::string path = "frame0001.png"; // here use your image
img_input = cv::imread(path);
img_gpu_in = cv::cuda::GpuMat(img_input); // build OpenCV GPU image based on the CPU version
// wrap an OpenCV GPU image around previously allocated output_buff
img_gpu_processed = cv::cuda::GpuMat(img_height_output, img_width_output, CV_32FC3, output_buff);
cv::cuda::resize(img_gpu_in, img_gpu_processed, cv::Size(img_height_output, img_width_output), 0, 0, cv::INTER_NEAREST);
// the resulting resized image is stored in output_buff and I can use it as model input in future

// now I try to do the same with VPIimage instead of OpenCV GPU version
VPIImage img_vpi;

VPIImageData data;
data.bufferType = VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR;
data.buffer.pitch.format = VPI_IMAGE_FORMAT_Y8_ER;
data.buffer.pitch.planes[0].data = output_buff;
data.buffer.pitch.planes[0].width = img_width_output;
data.buffer.pitch.planes[0].height = img_height_output;
data.buffer.pitch.planes[0].pixelType = VPI_PIXEL_TYPE_DEFAULT;

//VPIStatus err_vpi = vpiImageCreateWrapper(&data, nullptr, VPI_BACKEND_CUDA, &img_vpi);
VPIStatus err_vpi = vpiImageCreateWrapper(&data, nullptr, 0, &img_vpi);
std::cout << vpiStatusGetName(err_vpi) << std::endl; // Output: VPI_ERROR_INVALID_ARGUMENT

img_gpu_in.release();
img_gpu_processed.release();
cudaFree(output_buff);

return 0;

```

}

The error is still the same.

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [November 10, 2023, 6:32am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/15 "2023-11-10T06:32:51Z")

</div>

Hi,

Sorry for the late.  
Please see the sample below:

```auto
#include <iostream>
#include <fstream>
#include <sstream>

#define CHECK_STATUS(STMT) \
    do \
    { \
        VPIStatus status = (STMT); \
        if (status != VPI_SUCCESS) \
        { \
            char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
            vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
            std::ostringstream ss; \
            ss << "line " << __LINE__ <<": "; \
            ss << vpiStatusGetName(status) << ": " << buffer; \
            throw std::runtime_error(ss.str()); \
        } \
    } while (0);

int main(int argc, char *argv[])
{
    int width = 224;
    int height = 224;
    int pitch = (int((width-1)/256)+1)*256;

    char* ptr;
    cudaMalloc((void**)&ptr, pitch*height*sizeof(char));

    VPIImage img = NULL;
    VPIImageData data = {};
    data.bufferType = VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR;
    data.buffer.pitch.numPlanes = 1;
    data.buffer.pitch.planes[0].width = width;
    data.buffer.pitch.planes[0].height = height;
    data.buffer.pitch.planes[0].pitchBytes = pitch;
    data.buffer.pitch.planes[0].data = ptr;
    data.buffer.pitch.format = VPI_IMAGE_FORMAT_Y8_ER;

    CHECK_STATUS(vpiImageCreateWrapper(&data, nullptr, 0, &img));

    vpiImageDestroy(img);
    cudaFree(ptr);
    return 0;
}

```

Thanks.

---

<div class="post-metadata">

**Author:** ![valeriy.manin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@valeriy.manin](https://forums.developer.nvidia.com/u/valeriy.manin)\
**Post date:** [November 14, 2023, 2:13pm UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/16 "2023-11-14T14:13:02Z")

</div>

Thank you very much! The problem was that I didn’t use the pitch value. Could you please explain what is the idea behind this formula?  
int pitch = (int((width-1)/256)+1)\*256;

I understand that it is something with the memory layout. But what is the exact meaning and from where the 256 value comes from?

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [December 5, 2023, 2:08am UTC](https://forums.developer.nvidia.com/t/vpi-image-pointing-to-managed-memory/267775/19 "2023-12-05T02:08:19Z")

</div>

Closed on December 5, 2023, 2:08am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.