Optical flow tracking with Opencv Gpu Mat

Hi,
I want to use Optical Flow V2 to track object like in NvOFTSample.
To provide frame in input, I use OpenCV mat and GpuMat object. I use NvOFTracker class define in NvOFTracker.

In NvOFT_SURFACE_MEM_TYPE_SYSTEM Mode, I can track object without problem
In NvOFT_SURFACE_MEM_TYPE_CUDA_DEVPTR, the tracker do not update object bounding box until a new detection is done.
Seems that the input frame is ignored. I don’t have the same result when I use MEM_TYPE_SYSTEM and MEM_TYPE_CUDA_DEVPTR.

Here the code :

TrackedObjects Tracker::do_tracking(const cv::cuda::GpuMat& frame, const std::vector<NvOFT_OBJ_TO_TRACK> &objDetected, bool doDetection, bool reset) {
    cv::cuda::GpuMat tmpFrame = cv::cuda::createContinuous(frame.cols, frame.rows, CV_8UC1);
    cv::cuda::GpuMat resizeFrame = cv::cuda::createContinuous(_width, _height, CV_8UC1);
    cv::cuda::cvtColor(frame, tmpFrame, cv::COLOR_BGR2GRAY, 0, _cudaStream);
    cv::cuda::resize(tmpFrame, resizeFrame, cv::Size(_width, _height), 0, 0, cv::INTER_LINEAR, _cudaStream);

    _cudaStream.waitForCompletion();
    return _tracker.TrackObjects(resizeFrame.ptr<uint8_t>(), _width * _height, _width, objDetected, doDetection, reset);
}

Between 2 detections, bounding box of tracked object stay at the same position. When the objet moves a little bit faster, a new ID is associated to it. I didn’t have this issue when I use MEM_TYPE_SYSTEM and I give a Mat object instead of GpuMat

Hi L_JI.

Can you provide more details on the below points.

  1. The NvOFTSample by default use “NvOFT_SURFACE_MEM_TYPE_CUDA_DEVPTR” mem type. Do you see the same issue on NvOFTSample as well? If not already tried, could you try and post your observations?
  2. Are you initializing the tracker with the right mem type (that is “NvOFT_SURFACE_MEM_TYPE_SYSTEM” when you intend to use system memory and “NvOFT_SURFACE_MEM_TYPE_CUDA_DEVPTR” when intended use is gpu memory?)
  3. If not already done, could you dump the “resizeFrame” and make sure it is as expected?

Thanks.