VisionWorks: vxOpticalFlowPyrLKNode and vx_keypoint_t structure


I’m using the vxOpticalFlowPyrLKNode within our design. I started with the ‘Feature Tracker Demo App’ and modified it to use with our cameras. So far things are working well and I can see the features being tracked.

I have a couple of questions.

1. Question about the point structures

There are three different structures that can be populated by the feature tracker:

  • NVX_TYPE_POINT2F: X, Y values in float format
  • NVX_TYPE_KEYPOINTF: X, Y values in float format along with error, orientation, scale, strength and tracking_status
  • VX_TYPE_KEYPOINT: Similar to the previous except X and Y use integers instead of floats

I wanted more information than just the X and Y values so I chose to use VX_TYPE_KEYPOINT.

When I look at the data coming from both the current and previous points I see the ‘strength’ that represents the quality of the feature that was detected and the ‘tracking_status’ which, I have observed, is 1, 255 or 0.

what does the ‘255’ mean?

In the documentation it says that:

‘This function changes just the x, y, and tracking_status members of the vx_keypoint_t structure and behaves as if it copied the rest from the old tracking vx_keypoint_t to new image vx_keypoint_t.’

It doesn’t seem like the ‘error’, ‘orientation’ or ‘scale’ outputs do anything at the moment.

Is there someway of enabling the ‘error’?

2. How to handle lost features

From the 'Optical Flow Pyramid (LK) documentation:

‘Clients are responsible for editing the output vx_array of vx_keypoint_t structs array before applying it as the input vx_array of vx_keypoint_t structs for the next frame. For example, vx_keypoint_t structs with tracking_status set to zero may be removed by a client for efficiency.’

Looking at the feature tracker example in the demo directory the example doesn’t seem to edit the array of keypoints.

What happens if the keypoints are left in there?

If I wanted to remove the features with ‘0’ as the status I just remove them from the ‘current features’ and leave the ‘previous features’ alone. correct?

3. What is the most efficient way to move data around?

My application gets data from our camera board over PCI Express. I have a driver that exposes the camera information to a userland application.

This application copies camera data received over PCIE to one of three frame buffers. A populated frame buffer is then transferred to the feature tracking thread through a queue. I need to send the data from these frame buffers to the LK Tracker input vx_image. In order to accomplish this I’m using memcpy to copy the data from a frame buffer to the mapped vx_image that is then fed into the tracker as the new frame.

Is there a better way to do this?

Instead of copying the frame buffer data to the same location location in vx_image. is there a way to tell the graph that the new frame is at a new location specified by a pointer?

Thanks for any help,



The definition of strength is specific to the corner detector. Please check the detector document for information.
For example, vxFastCornersNode
Corner Strength Computation

Once a corner has been detected, its strength (response, saliency, or score) shall be computed if nonmax_suppression is set to true, otherwise the value of strength is undefined. The corner response Cp function is defined as the largest threshold t for which the pixel p remains a corner.

You can update the vx_keypoint_t based on the requirement of your use case.
Please modify the current features and the previous features will be overwritten once the new one arrived.

We have some sample for zero copy camera source pipeline.
Please check our MMAPI for more information.