I am following the ROS1/Cameras tutorial in order to create a camera publisher with 3D bounding box data.
I have a simple scene with a cylinder for which I have attached a semantic label. In the viewport, I am able to see the semantic segmentation and 2D or 3D bounding boxes.
However, the ROS messages are not working properly.
In ROS1 Camera Helper, I have tried the following types:
- Semantic segmentation
rostopic list, I see the topic /semantic (✓)
rostopic echo /semantic, I see a bunch of zeros and ones (seems ok)
- In rviz though, I get unsupported image encoding [32SC1]
- 2D bounding box
rostopic list, I see the topic /bbox_2d (✓)
rostopic echo /bbox_2d, I get:
ERROR: Cannot load message class for [vision_msgs/Detection2DArray]. Are your messages built?
- 3D bounding box
- No topic is declared at all. (
rostopic list does not show the topic)
rgb and depth types are fine and I can visualize them in rviz.
Could you please let me know how to create a valid 3D bounding box camera publisher?
Thank you so very much for your response!
After installing the
vision-msgs package, both the 2D and 3D bounding box messages work!
Also, using your script, I was able to visualize the semantic segmentation in rviz as well. Thank you very much for sharing!
I have been looking at the 3D bounding box data but it seems that something is off.
I wasn’t sure whether the pose of the bounding box is w.r.t. the camera or the world (actual pose equal to the prim’s pose shown in properties) but after moving the camera, I get the same pose for the bounding boxes, so I assume it is w.r.t. the world. Is this accurate?
The issue I have is that the bounding boxes seem accurate only for cubes and not for spheres or cylinders (haven’t tested anything else).
In the .usd example given above, I have added a cylinder and translated the cube.
For the cube, the center of the bbox matches the cube’s center, there is no rotation and the size is correct:
However, for the cylinder, the bbox’s center position is not the same as the prim’s translation, it shows a rotation that does not exist and the size also seems wrong.
I would expect to get:
- position: x:3.26, y:0, z:0
- orientation: x:0, y:0, z:0, w:1
- size: x:1, y:1, z:1
Same for the sphere.
The center’s position numbers are the same but they refer to different axes, there is a rotation that shouldn’t exist and the size is different.
Since the cube’s bbox matches the prim’s characteristics, I would expect that to happen for the other objects as well. So, are the other data indeed wrong? If not, how should I interpret them?
Edit: After examining the bbox data in another scene with two cubes and a cylinder, I get “correct” results that match the prim’s attributes for the cylinder and “wrong” results for the cubes. So, it is not the shape that matters but I cannot figure out why only one shape gives “correct” results while the others don’t.
As a user, I lack sufficient information to answer these questions.
@Hammad_M could you please help us identify what might be wrong?
Oh, I see. Thank you very much for replying though!
Due to a memory layout change bbox2d loose, tight and bbox3d were all incorrect in 2022.2.0
I have added better tests to catch this and it is fixed in 2022.2.1, coming out this week (fingers crossed)
tested with the warehouse and ROS2 humble and it is correct in 2022.2.1:
Oh, okay, so I will wait for the new version release and hopefully, it will work with ROS 1 as well.
Thank you for your response!
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.