Get Orientation Pointcloud of objects in camera pixel


In the YCB_Video Dataset there is a variable that is called “vertmap”. It has the shape (640,480,3) (pixel_x,pixel_y,point(x,y,z)) and shows the distance from the object center (3D) to the shown point in the image. So it is not a real pointcloud that has its center in the camera, but for each object it has its own point cloud that originate from the center of said object. Is there any internal function that can do something like this? Or should I start from scratch? My goal is to generate a dataset really close to the YCB_Video dataset, but with my own objects.

If you have questions to understand what I mean, feel free to ask. I will attach a vertmap.npy and a png file, so if you want to further understand feel free to download.
000001-meta.mat (598.6 KB)
000001-vertmap.npy (3.5 MB)
000001-box.txt (123 Bytes)

Thank you!

Hi @valentinhendrik , have you checked the replicator’s pointcloud annotator? It should gives you the desired pointcloud. But that requires you to load your own asset and generate the data. You can take a look at which uses To get the poincloud data, you need to add a arg that sets pointcloud=True

Yes I know that, but it is not using a regular pointcloud. What I need is a mapping of points in object coordinate system in the image array. Pointcloud annotator only gives out points in world coordinates.

In that case, you can use the CameraParams annotator to get the cameraProjectionMatrix and cameraViewTransform matrix to convert the world points to object space, then to image space.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.