VisionWorks example importing Point Grey FlyCapture camera image buffer into processing graph

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX1
1

Hello,

Would someone please point me to some example code for getting an external image buffer into a vx_image? I have a PointGrey USB 3.0 camera, and I’m grabbing images using the FlyCapture library and SDK. After I retrieve a point grey image buffer, what is the best way to get that buffer into the VisionWorks processing chain?

Thanks

2

I read they don’t support USB camera’s right now ?

3

Do they support putting raw data into the Vision Works pipeline? I can use the FlyCapture lib to read the USB 3.0 camera frames off of the camera. I’m having trouble figuring out how to get this data, which has already been read off the camera, into the Vision Works API.

4

I managed to get the USB 3.0 camera data into the VisionWorks pipeline, but I have no idea if this is an efficient way of doing this. In summary, I setup the camera, grab the frame on each iteration, and copy it into the src2 image which was setup to process frame. I’m using the VisionWorks sample program opencv_npp_interop as a starting point.

...
   vx_imagepatch_addressing_t src2_addr;
   src2_addr.dim_x = cv_src2.cols;
   src2_addr.dim_y = cv_src2.rows;
   src2_addr.stride_x = sizeof(vx_uint8);
   src2_addr.stride_y = static_cast<vx_int32>(cv_src2.step);

   void *src2_ptrs[] = {cv_src2.data};

   vx_image src2 = vxCreateImageFromHandle(context, VX_DF_IMAGE_U8, &src2_addr, src2_ptrs, VX_IMPORT_TYPE_HOST);
   NVXIO_CHECK_REFERENCE(src2);
...
  void *src2_ptr = NULL;
  NVXIO_SAFE_CALL( vxAccessImagePatch(src2, &rect, 0, &src2_addr, &src2_ptr, VX_READ_ONLY) );
  NVXIO_SAFE_CALL( vxCommitImagePatch(src2, NULL, 0, &src2_addr, src2_ptr) );
...
  vx_node blur2_node = vxGaussian3x3Node(graph, src2, src2_blurred);
  NVXIO_CHECK_REFERENCE(blur2_node);

...
while(1) {
...
   cam_image_ptrs[0] = image_right.GetData();

   vxReleaseImage (&cam_image);
   cam_image = vxCreateImageFromHandle(context, VX_DF_IMAGE_U8, &src2_addr, cam_image_ptrs, VX_IMPORT_TYPE_HOST);
   NVXIO_CHECK_REFERENCE (cam_image);

   void *cam_image_ptr = NULL;
   NVXIO_SAFE_CALL( vxAccessImagePatch(cam_image, &rect, 0, &src2_addr, &cam_image_ptr, VX_READ_ONLY) );
   NVXIO_SAFE_CALL( vxCommitImagePatch(cam_image, NULL, 0, &src2_addr, cam_image_ptr) );
   NVXIO_SAFE_CALL( nvxuCopyImage(context, cam_image, src2));
...
}
5

I believe you can supply your own buffers to FlyCapture 2. If visionworks can allocate some memory that it is prepared / optimized to use you could pass that to FlyCap and avoid at least one memcpy.