I cannot get point cloud data at RL environment

Omniverse Isaac Sim
1

Hi,

There is a problem with getting point cloud on a new version, 2023.

[Problem 1]
[omni.isaac.range_sensor.plugin] Lidar Sensor does not exist
The error message occurred and, I found that the error message appeared from

def warm_start(self):
    ....
    self._physx_interface.update_simulation(self.get_physics_dt(), 0.0)

at exts/omni.isaac.core/omni/isaac/core/physics_context/physics_context.py.
kit_20231117_031816.log (685.2 KB)

It occurred when I trained my RL code which worked well on ver.2022. You can check my attachment. I will work on OmniIsaacGymEnvs and I left only my custom files.
omniisaacgymenvs.zip (12.1 MB)

Thanks.
.
.

[Problem 2] SOLVED!
There is a point cloud segmentation example in the documentation (link). I modified it to standalone version (below attachment) and get the same results with documentation on ver.2022. However, it seems not working well on the new version and prints warning messages.

2023-11-16 18:55:18 [16,435ms] [Warning] [omni.isaac.range_sensor.python] Lidar get_semantic_data is deprecated and will not return any data, use get_prim_data and access semantics via usd

Anyway, I can get point cloud data even though the warning message appears but, I cannot get semantics.

(Please change .txt to .py)
point_cloud_seg.txt (3.5 KB)

[2023-11-17 added]
I can solve the second question by referring to documentation (link). I think the documentation has to be changed.

1 Like
2

I think the prim_path is not the reason for this problem.

I guess the simulator did not find any Lidar Sensor so if it has an empty np.ndarray value then it cannot convert the value.

I also checked the entire prim_path via the code below and then, entered the Lidar-related path but, it did not work.

stage = omni.usd.get_context().get_stage()

# Check if stage is valid
if stage:
# Iterate over all prims in the stage
for prim in stage.TraverseAll():
    # Print the prim path
    print(prim.GetPath())

@toni.sm @ahaidu I’m sorry, but can you shed some light on this issue?

4

I also tried not just using Lidar, which existed on USD, but also creating new Lidars. However, it does not work too.

@rthaker, @kellyg Would you mind checking the issue above?

There is a Lidar at each environment but, the error occurred.

image
image1920×985 186 KB

Additionally, here is a suggestion about modifying rotating_lidar_physX.py at _data_acquisition_callback.

if key == "semantics":
### original code ###
# self._current_frame[key] = self._backend_utils.create_tensor_from_list(
#     self._lidar_sensor_interface.get_semantic_data(self.prim_path),
#     dtype="float32",
#     device=self._device,
# )
### original code ###
self._current_frame[key] = self._backend_utils.create_tensor_from_list(
    self._lidar_sensor_interface.get_prim_data(self.prim_path),
    dtype="float32",
    device=self._device,
)
else:
### original code ###
# self._current_frame[key] = self._backend_utils.create_tensor_from_list(
#     getattr(self._lidar_sensor_interface, "get_{}_data".format(key))(self.prim_path),
#     dtype="float32",
#     device=self._device,
# )
### original code ###
self._current_frame[key] = self._backend_utils.create_tensor_from_list(
    getattr(self._lidar_sensor_interface, "get_{}_data".format(key))(self.prim_path).astype(dtype='int32'),
    dtype="float32",
    device=self._device,
)

  1. From 2023, get_semantic_data is deprecated. I think it should be changed to get_prim_data.

  2. If there are any point cloud data, the type is assigned into numpy.unit16. It caused the shutdown Isaac Sim. I think it would be better to convert it into another data type.

7

Is there anyone who can help with this problem?