Problem with SparsePyrLKOpticalFlow and CUDA

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hi,
I’m trying to test different ways to get the keypoints to use in SparsePyrLKOpticalFlow.
My code at the moment more or less is the following:
std::vectorcv::KeyPo int> keypoints;

cv::Ptr<cv::cuda::FastFeatureDetector> p = cv::cuda::FastFeatureDetector::create();
p->detect(prev_grey, keypoints);
std::cout << "keypointsSize: " << keypoints.size() << std::endl;
cv::cuda::GpuMat test2(1, keypoints.size(), CV_32FC2, static_cast<void *>(keypoints.data()));
std::cout << "created: " << test2.cols << std::endl;
cv::Ptr<cv::cuda::SparsePyrLKOpticalFlow> d_pyrLK_sparse = cv::cuda::SparsePyrLKOpticalFlow::create(cv::Size(win_size, win_size), max_level);
std::cout << "info: " << test2.type() << "; " << test2.cols << "; " << test2.rows << std::endl;
d_pyrLK_sparse->calc(use_gray ? prev_grey : prev_frame_resize, use_gray ? cur_grey : cur_resize, test2, d_next_pts, d_status);

When I use this code on another Windows PC it works perfectly, but when I try to use it on my Jetson Xavier AGX I get the next error:

OpenCV(4.4.0) /home/nvidia/opencv_contrib 4.4.0/modules/cudev/include/opencv2/cudev/grid/detail/transform.hpp:267: error: (-217:Gpu API call) an illegal memory access was encountered in function ‘call’

Another detail is that if I use the following code:

cv::Ptrcv::cuda::CornersDetector detector = cv::cuda::createGoodFeaturesToTrackDetector(prev_grey.type(), max_corners, quality_level, min_dist);
detector->detect(prev_grey, d_prev_pts);
cv::Ptrcv::cuda::SparsePyrLKOpticalFlow d_pyrLK_sparse = cv::cuda::SparsePyrLKOpticalFlow::create(cv::Size(win_size, win_size), max_level);
d_pyrLK_sparse->calc(use_gray ? prev_grey : prev_frame_resize, use_gray ? cur_grey : cur_resize, d_prev_pts, d_next_pts, d_status);

The program works correctly. I don’t understand the error because in both cases I use a GpuMat.

Thanks in advance

3

Hi,
We have tried cv::cuda::createSobelFilter in the sample code:

could you check if SobelFilter works in your environment? Would like to know if it is specific to SparsePyrLKOpticalFlow APIs.

4

Hi,
With the sobelFilter I don’t have any problem, what’s wrong with SparsePyrLKOpticalFlow??
I don’t understand what I’m doing wrong.

5

Hi,
The buffer format is RGBA in calling createSobelFilter(). Probably the format is not supported in SparsePyrLKOpticalFlow functions?

7

I had a similar issue. I had to manually convert the keypoints to Point2f:

    FAST->detect(d_prev, gpu_keypoints);

	// there must be a better way of doing this...
	std::vector<cv::Point2f> points;
	std::vector<cv::KeyPoint>::iterator it;

	for (auto it = gpu_keypoints.begin(); it != gpu_keypoints.end();it++)
		points.push_back(it->pt);

	d_prev_keypoints.upload(points);

	LK_OF->calc(
				d_prev, d_current,
				d_prev_keypoints, d_current_keypoints);
1 Like