ROS propper way to get camera pointcloud and jointState at the same time

Omniverse Isaac Sim
,
1

Hi,
I’ve got scene with robotic arm and camera mounted on end-effector in Isaac sim. I’m using standalone application and ROS clock to provide simulation time. I want to get exact joints position at which camera acquired an image. (and than do FK to obtaion exact camera position so I can transform camera pointcloud).

Currently I have ros node which subscribes to camera and jointstate and subscribers callback just save message to memory. In case service for getting image with jointstate is called I response with latest image and jointstate saved to the memory.
Unfortunatelly, the transformed pointcloud is shaking (and I think it’s because of different time of image and jointstate). Is there any solution how to get exact jointstate in which camera acquired and image? Thank you.

EDIT: for better explanation I add video of transformed pointcloud. Pointcloud should not be moving at all in world coordinates.

Camera point cloud shake when rotate
Camera images are always delayed 2 rendering frames
2

yes , We both have the same problem

3

hey bro , have you solved it ?

4

@email5 Can you publish the pose of the camera directly via the PoseTree publisher component?
That would remove FK as a variable and the pose tree would contain the exact world pose of the camera at the same timestamps as the camera data itself.

5

Thank you for your answer. I find out that my point cloud was big and with a fast publish rate and rViz cannot handle such a large data. I republished the point cloud with:

<node name="issac_pointcloud_throttle" pkg="topic_tools" type="throttle" args="messages /point_cloud 10 /point_cloud_throttle" />

I don’t have a node that transforms the point cloud, I set frame_id to camera and publish static transformer from the robot’s end-effector (published by Isaac sim Ros PoseTree) to the camera.

But I’ve got another problem. In my application, I need a service, which returns an image and corresponding transformation. (pose from which the image was acquired). I wrote a node, which subscribes to Isaac sim image and joint_state. To sync them I use message_filter::Syncrhonizer with ExactTime policy. Unfortunately, even if image and joint_state have the same timestamp in the header, joint_state does not match the exact position from which the image was taken. (and it depends on how fast the robot moves - when it’s moving fast, the position is more wrong)

6

I am using latest Isaac Sim (2022.02.1). I found that the rendered results are always delayed 2 rendering frames. This problem should be fixed since wrong timestamp makes whole system unreliable.