In the YCB_Video Dataset there is a variable that is called “vertmap”. It has the shape (640,480,3) (pixel_x,pixel_y,point(x,y,z)) and shows the distance from the object center (3D) to the shown point in the image. So it is not a real pointcloud that has its center in the camera, but for each object it has its own point cloud that originate from the center of said object. Is there any internal function that can do something like this? Or should I start from scratch? My goal is to generate a dataset really close to the YCB_Video dataset, but with my own objects.
If you have questions to understand what I mean, feel free to ask. I will attach a vertmap.npy and a png file, so if you want to further understand feel free to download. 000001-meta.mat (598.6 KB) 000001-vertmap.npy (3.5 MB) 000001-box.txt (123 Bytes)
Hi @valentinhendrik , have you checked the replicator’s pointcloud annotator? It should gives you the desired pointcloud. But that requires you to load your own asset and generate the data. You can take a look at offline_pose_generation.py which uses ycb_video_writer.py. To get the poincloud data, you need to add a arg that sets pointcloud=True
Yes I know that, but it is not using a regular pointcloud. What I need is a mapping of points in object coordinate system in the image array. Pointcloud annotator only gives out points in world coordinates.
In that case, you can use the CameraParams annotator to get the cameraProjectionMatrix and cameraViewTransform matrix to convert the world points to object space, then to image space.
