Editing data on a VPIArray

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
#1

Hi,
I want to modify the Pyramidal LK Optical Flow VPI Sample to fill my needs.
I only want to use the Optical flow sample, I don’t need Harris point detector as i already have a payload containing keypoints. I can’t find a way to enter my keypoints payload in the optical flow function : vpiSubmitOpticalFlowPyrLK()
I need to edit the data in a VPIArray, but i didn’t find how to do it. I can read the data using

vpiArrayLock(arrPrevPts, VPI_LOCK_READ_WRITE, &prevPtsData);

and get the value in the VPIArrayData prevPtsData. Writting on a VPIArrayData with VPILockMode VPI_LOCK_READ_WRITE and VPI_LOCK_WRITE doesn’t seems to write data in the VPIArray.
I tried the vpiArraySetWrappedHostMem function but didn’t get any result.

I just need to regularly send keypoints into the optical flow algorithm.
Thank you in advance.

#3

Hi,

Here is a sample to construct a pre-defined VPIKLTTrackedBoundingBox into a VPIArrayData.
You can follow the same way for VPI_ARRAY_TYPE_KEYPOINT.

Thanks.

#4

Hi AastaLLL,
Thanks for your reply,
I already know how to write data in a VPIArrayData with

arraydata.data = (VPIKeypoint *) cv::Point(x,y);

I want to use the optical flow function vpiSubmitOpticalFlowPyrLK() in a loop and have access to his data, but that function only takes a VPIArray as a input argument containing keypoint.
So i know how to modify VPIArrayData but i don’t know how to push VPIArrayData into VPIArray

#5

Hi,

You can wrap a VPIArray from VPIArrayData.
https://docs.nvidia.com/vpi/group__VPI__CPUInterop.html#gaeb960deaf4fbf858bf9a634afd274c9b

For example, in the sample shared above:

VPIArrayData data = {};
...
CHECK_STATUS(vpiArrayCreateHostMemWrapper(&data, 0, &inputBoxList));

Thanks.

#6

Hi AastaLLL,
Thanks for fast reply.
I succeeded with the example that you shared, now i can edit data in VPIArray. May i ask why the code #1 work but not the second ? Can you please point out my mistake ?
Code 1 :

arraydata.format = VPI_ARRAY_TYPE_KEYPOINT;
arraydata.capacity = MAX_KEYPOINTS;
arraydata.sizePointer = &posize;
arraydata.data = (VPIKeypoint *) &matrix_containing_point; // (([530, 260] … ))
CHECK_STATUS(vpiArrayCreateHostMemWrapper(&arraydata, 0, &arrPrevPts3));
VPIArrayData prevPtsData;
CHECK_STATUS(vpiArrayLock(arrPrevPts3, VPI_LOCK_READ_WRITE, &prevPtsData));
cv::Point *pPrevPts = (cv::Point *)prevPtsData.data;
pPrevPts[0].x; //output 530
CHECK_STATUS(vpiArrayUnlock(arrPrevPts3));

Code 2 :

arraydata.format = VPI_ARRAY_TYPE_KEYPOINT;
arraydata.capacity = MAX_KEYPOINTS;
arraydata.sizePointer = &posize;
arraydata.data = (VPIKeypoint *) &matrix_containing_point; // (([530, 260] … ))
CHECK_STATUS(vpiArrayCreateHostMemWrapper(&arraydata, 0, &arrPrevPts3));
VPIArrayData prevPtsData;
CHECK_STATUS(vpiArrayLock(arrPrevPts3, VPI_LOCK_READ_WRITE, &prevPtsData));
VPIKeypoint *pPrevPts = (VPIKeypoint *)prevPtsData.data;
pPrevPts[0].x; //Ouput x^(-43) Can’t find a way to get the correct value
CHECK_STATUS(vpiArrayUnlock(arrPrevPts3));

Thank you for helping me.

#7

And what if i want to reuse a VPIArray to entirely edit his data but keep his type, capacity, etc.

VPIArray1 contains data
CHECK_STATUS(vpiSubmitHarrisCornerDetector(stream, 0, VPIPayload, VPIImage, VPIArray1, VPIArray2, &harrisParams)); //I want this to replace data from the VPIArray1

I want to do something like that, this doesn’t seems to work.

#8

Hi,

We cannot reproduce the issue of Code 2.
The sample is modified it from the VPI sample in /opt/nvidia/vpi1/samples/03-harris_corners.

{
    // Lock output keypoints and scores to retrieve its data on cpu memory
    VPIArrayData outKeypointsData;
    VPIArrayData outScoresData;
    CHECK_STATUS(vpiArrayLock(keypoints, VPI_LOCK_READ, &outKeypointsData));
    CHECK_STATUS(vpiArrayLock(scores, VPI_LOCK_READ, &outScoresData));

    VPIKeypoint *outKeypoints = (VPIKeypoint *)outKeypointsData.data;
    uint32_t *outScores       = (uint32_t *)outScoresData.data;
    std::cout << "keypoint x:" << outKeypoints[0].x << std::endl;
    ...
}

We can read the output normally without issue.

$ ./vpi_sample_03_harris_corners cuda ../assets/kodim08.png

NVMEDIA_ARRAY: 53, Version 2.1
NVMEDIA_VPI : 172, Version 2.4
keypoint x:495
267 keypoints found

Thanks.

#9

Hi AastaLLL,
The sample works for me too, i get the exact same result as you. Here is a code extract that illustrates what i wanted to say and it gets wrong values when i try to cast VPIKeypoints <–> cv::Point :

        CHECK_STATUS(vpiArrayCreate(MAX_KEYPOINTS, VPI_ARRAY_TYPE_KEYPOINT, 0, &arrPrevPts4));
		VPIArrayData arraydata;
		int32_t posize = 300;
		cv::Point matr[2][2] = {{{21,22}, {31,32}}, {{41,42}, {51,52}}}; // Line A
        //VPIKeypoint matr[2][2] = {{{21,22}, {31,32}}, {{41,42}, {51,52}}}; // Line B

		arraydata.format = VPI_ARRAY_TYPE_KEYPOINT;
		arraydata.capacity = MAX_KEYPOINTS;
		arraydata.sizePointer = &posize;
		arraydata.data = (VPIKeypoint *) &matr[0][0];   //Work

		CHECK_STATUS(vpiArrayCreateHostMemWrapper(&arraydata, 0, &arrPrevPts4));

		VPIArrayData prevPtsData;
		CHECK_STATUS(vpiArrayLock(arrPrevPts4, VPI_LOCK_READ, &prevPtsData));

		VPIKeypoint *pPrevPts = (VPIKeypoint *)prevPtsData.data;
		cv::Point *pPrevPts_point = (cv::Point *)prevPtsData.data;

		std::cout << "pPrevPts[0].x, pPrevPts[0].y : " << pPrevPts[0].x << "  ,  " << pPrevPts[0].y << std::endl;
		std::cout << "pPrevPts_point[0].x, pPrevPts_point[0].y : " << pPrevPts_point[0].x << "  ,  " << pPrevPts_point[0].y << std::endl;

		CHECK_STATUS(vpiArrayUnlock(arrPrevPts4));

Output when line A is on :
pPrevPts[0].x, pPrevPts[0].y : 2.94273e-44 , 3.08286e-44
pPrevPts_point[0].x, pPrevPts_point[0].y : 21 , 22

Output when line B is on :
pPrevPts[0].x, pPrevPts[0].y : 21 , 22
pPrevPts_point[0].x, pPrevPts_point[0].y : 1101529088 , 1102053376

I can’t figure out what i did wrong.

#10

Hi,

This is make sense.
Since vpiArrayCreateHostMemWrapper is just a wrapper from the input buffer.

If you feed a cv::Point buffer(line A).
You will need to cast the output data into cv::Point (pPrevPts_point).

If you feed a VPIKeypoint buffer(line B).
To cast the output data into VPIKeypoint is necessary (pPrevPts).

But if you try to run other VPI algorithm, it’s expected to use VPIKeypoint buffer.
Please implement a converter to change the format from cv::Point → VPIKeypoint.

Thanks.

#11

Hi AastaLLL,
Indeed, the problem comes from the cast from those types to another cv::Point <–> VPIKeypoint . But i thought it would do it automatically since they both only contains x and y, but it’s fine i found another solution to this.
Thanks for your time and the help provided !

#12

Hi,

The structure between VPIKeypoint and cv::Point is quite different.

In VPIKeypoint: VPI - Vision Programming Interface: Common Types

 typedef struct
{
    float x, 
    float y;   
} VPIKeypoint;

In cv::Point: opencv/types.hpp at master · opencv/opencv · GitHub

template<typename _Tp> class Point_
{
public:
    ...
    _Tp x; //!< x coordinate of the point
    _Tp y; //!< y coordinate of the point
};

Good to know you fix this already.
Above information is just for your reference.

Thanks.

1 Like
#14