Isaac 2023.1.0 examples slow and buggy

Omniverse Isaac Sim
1

Hi all,

I’m having some hiccups after updating to 2023.1.0. Some examples are very slow to launch and kind of buggy. Especially regarding motion planning and Lula.

Now throw some more warnings like:

[…]

2023-10-18 20:36:34 [14,436ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!

2023-10-18 20:36:46 [25,928ms] [Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version:

and
[…]

[Lula] Joint 'left_inner_finger_joint' is specified as a mimic joint, but its control chain ['left_inner_finger_joint' -> 'finger_joint'] terminates with a joint ['finger_joint'] that is not a c-space coordinate. Mimic attributes will be ignored.
[Lula] Joint 'left_inner_knuckle_joint' is specified as a mimic joint, but its control chain ['left_inner_knuckle_joint' -> 'finger_joint'] terminates with a joint ['finger_joint'] that is not a c-space coordinate. Mimic attributes will be ignored.
[Lula] Joint 'right_inner_finger_joint' is specified as a mimic joint, but its control chain ['right_inner_finger_joint' -> 'finger_joint'] terminates with a joint ['finger_joint'] that is not a c-space coordinate. Mimic attributes will be ignored.
[Lula] Joint 'right_inner_knuckle_joint' is specified as a mimic joint, but its control chain ['right_inner_knuckle_joint' -> 'finger_joint'] terminates with a joint ['finger_joint'] that is not a c-space coordinate. Mimic attributes will be ignored.
[Lula] Joint 'right_outer_knuckle_joint' is specified as a mimic joint, but its control chain ['right_outer_knuckle_joint' -> 'finger_joint'] terminates with a joint ['finger_joint'] that is not a c-space coordinate. Mimic attributes will be ignored.

and

Throws this error and crashes, after stopping the simulation and starting again:

2023-10-18 20:40:19 [238,845ms] [Error] [omni.isaac.motion_generation.articulation_motion_policy] Attempted to compute an action, but the robot Articulation has not been initialized. Cannot get joint positions or velocities.

Exception has occurred: AttributeError
'NoneType' object has no attribute 'astype'
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/motion_policies.py", line 510, in _evaluate_acceleration
    joint_positions = joint_positions.astype(np.float64)
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/motion_policies.py", line 503, in _euler_integration
    joint_accel = self._evaluate_acceleration(joint_positions, joint_velocities)
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/motion_policies.py", line 489, in _update_robot_joint_states
    self._robot_joint_positions, self._robot_joint_velocities = self._euler_integration(
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/motion_policies.py", line 148, in compute_joint_targets
    self._update_robot_joint_states(active_joint_positions, active_joint_velocities, frame_duration)
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/articulation_motion_policy.py", line 102, in get_next_articulation_action
    position_targets, velocity_targets = self.motion_policy.compute_joint_targets(
  File "/home/user/.local/share/ov/pkg/isaac_sim-2023.1.0/standalone_examples/api/omni.isaac.manipulators/rmpflow_supported_robots/supported_robot_follow_target_example.py", line 122, in <module>
    actions = articulation_rmpflow.get_next_articulation_action()
AttributeError: 'NoneType' object has no attribute 'astype'

Is there any new documentation I should refer to?

I’m in Kubuntu 22.04, RTX 3080 Laptop 16GB VRAM. Driver Version: 525.125.06 CUDA Version: 12.0

Thanks!

1 Like
2

@edsonbffilho Can you say in which example you are seeing this error?

3

arudich will be looking into your issues deeper.

I wanted to take a moment to explain the mimic joint warning. With the Isaac Sim 2023.1.0 release, we have added support for mimic joints in Lula (forward kinematics, inverse kinematics, RMPflow, etc.). It looks like your URDF has the finger and knuckle joints all linearly mapped from finger_joint. Lula now supports the ability to respect this mapping. To support the mapping, finger_joint would need to be added as a c-space coordinate in the robot description YAML file supplied to Lula.

If finger_joint was included as a c-space coordinate, this would increase the number of c-space of coordinates for the robot by one and 6 joints would be added to Lula’s internal kinematic structure (where the 5 mimic joints would be controlled according to the linear mapping in the URDF). This would enable, for example, controlling a finger tip frame with RMPflow in a way that respects the linear map between knuckle and finger joints.

Since finger_joint is not defined as c-space coordinate, Lula is just letting your know that it found mimic attributes in the URDF that are being ignored. In this case, the URDF will produce Lula kinematics equivalent to previous releases. If you wanted the warning to go away, you could remove the mimic attributes for the joints from your URDF with no change to behavior.

4

For example, the replay follow target extension

5

@arudich @bpeele Is there a fix to the issue related to the warning

2023-10-19 20:25:54 [24,106ms] [Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version:

Right now it is taking around 200 seconds to load a program. Even the ones in the standalone_example take a long time to load.

6

Hi @bolun, generally that warning can be ignored but if you are seeing a long wait, there could be an issue looking for the assets. Can you confirm you have local Nucleus installed and can access it?

7

@Sheikh_Dawood Thanks for the response.

I attached a screenshot. Does this mean that I have Nucleus installed and accessible?

Screenshot from 2023-10-19 18-14-05
Screenshot from 2023-10-19 18-14-051853×1054 104 KB

8

That looks like you have the latest Nucleus installed but it does not show that you are logged into the server.

Can you try logging in to this page on your browser?

http://localhost:34080/omniverse://localhost/NVIDIA/Assets/Isaac/2023.1.0/

Also check that all services are green here:
http://localhost:3080/

9

@Sheikh_Dawood Thank you! After logging in it now loads much faster!

Another issue I have is when running code it shows me this error multiple times

2023-10-19 22:30:29 [7,015ms] [Error] [omni.ext._impl.custom_importer] Failed to import python module omni.isaac.lula. Error: /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liblula_kinematics.so: undefined symbol: _ZN4urdf9parseURDFERKSs.

Do you know what might be causing this? I can create a new thread with my code included if that helps.

10

Great to here it helps! We will take a look at the fix for that warning in the next version. Tracking it as OM-112464.

@arudich @bpeele Could help with the lula error.

11

I was having the same warning when calling utils.nucleus.get_assets_root_path() in 2023.1.0:

[omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version

After setting up a Nucleus Service on my localhost and logging in, I was able to successfully retrieve the assets path. What prompted this change from the previous version? I didn’t need to do any of these steps in 2022.2.1.

12

Hi. I believe there is a change in omni.client that has a longer timeout (compared to the previous version) if a Nucleus could not be reached. We are currently using a non asynchronous call to check for the Nucleus server. Switching it to an async call should fix this warning.

Note that this timeout should not happen if you have local Nucleus installed or have set the default asset root flag to an accessible remote Nucleus.

2 Likes
13

@bolun What does this command give you?

ldd /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liblula_kinematics.so
14

For anyone wondering, a mimic joint is a joint that automatically follows the value of another joint, with a possible multiple and offset. So if “x” is the angle of one joint, a mimic joint can have the value “y” set automatically, where y=multiple*x + offset

15

@BuckBabich The output is

╰─ ldd /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liblula_kinematics.so
	linux-vdso.so.1 (0x00007ffd427a4000)
	liblula_math.so => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liblula_math.so (0x00007f17a5c84000)
	liblula_util.so => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liblula_util.so (0x00007f17a5a26000)
	liburdfdom_model.so.3.0 => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/liburdfdom_model.so.3.0 (0x00007f17a57fc000)
	libyaml-cpp.so.0.6 => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/libyaml-cpp.so.0.6 (0x00007f17a555c000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f17a5335000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f17a51e4000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f17a51c9000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f17a4fd7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f17a617f000)
	libconsole_bridge.so.1.0 => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/libconsole_bridge.so.1.0 (0x00007f17a4dd1000)
	libtinyxml.so => /home/mocap/.local/share/ov/pkg/isaac_sim-2023.1.0/exts/omni.isaac.lula/pip_prebundle/_lula_libs/libtinyxml.so (0x00007f17a4ba3000)
16

@bolun That looks okay. When you say “running code,” is that code in an extension, or are you running standalone (i.e., via SimulationApp)?

What other modules is that code using?

Similar to Segmentation Fault after creating Pinocchio robot , the error you’re seeing suggests that you have conflicting versions of liburdfdom_model.so and the wrong one is getting loaded at runtime.

17

@edsonbffilho I looked at the Replay Follow Target example in omni.isaac.examples, but I was not able to cause any errors. Is this the extension that you meant? The rest of your post seems to be referring to a standalone script, but there is not a standalone script that runs the Replay Follow Target example.

18

Please that a look at this thread for a detailed description: Lula and dofbot errors.

19

hi, I also have the problem:[Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version

I combined the standalone_expamble (clone_ants.py and livestream.py) to test, the whole code is:

import argparse
from omni.isaac.kit import SimulationApp

parser = argparse.ArgumentParser("Omniverse Robotics Environments!")
parser.add_argument("--headless", action="store_true", default=False, help="Force display off at all times.")
args_cli = parser.parse_args()
config = {"headless": args_cli.headless}
if args_cli.headless is True:
    config = {
        "width":            1280,
        "height":           720,
        "window_width":     1920,
        "window_height":    1080,
        "headless":         args_cli.headless,
        "renderer":         "RayTracedLighting",
        "display_options":  3286,
        "sync_loads":         False,
    }

simulation_app = SimulationApp(launch_config=config)
if args_cli.headless is True:
    simulation_app.set_setting("/app/window/drawMouse", False)
    simulation_app.set_setting("/app/livestream/proto", "ws")
    simulation_app.set_setting("/app/livestream/websocket/framerate_limit", 120)
    simulation_app.set_setting("/ngx/enabled", False)
import sys

import carb
import numpy as np
from omni.isaac.cloner import GridCloner
from omni.isaac.core import World
from omni.isaac.core.articulations import ArticulationView
from omni.isaac.core.utils.nucleus import get_assets_root_path
from omni.isaac.core.utils.stage import add_reference_to_stage, get_stage_units

assets_root_path = get_assets_root_path()
if assets_root_path is None:
    carb.log_error("Could not find Isaac Sim assets folder")
    simulation_app.close()
    sys.exit()

my_world = World(stage_units_in_meters=1.0)
my_world.scene.add_default_ground_plane()

# create initial robot
asset_path = assets_root_path + "/Isaac/Robots/Ant/ant.usd"
add_reference_to_stage(usd_path=asset_path, prim_path="/World/Ants/Ant_0")

# create GridCloner instance
cloner = GridCloner(spacing=2)

# generate paths for clones
target_paths = cloner.generate_paths("/World/Ants/Ant", 4)

# clone
position_offsets = np.array([[0, 0, 1]] * 4)
cloner.clone(
    source_prim_path="/World/Ants/Ant_0",
    prim_paths=target_paths,
    position_offsets=position_offsets,
    replicate_physics=True,
    base_env_path="/World/Ants",
)
# create ArticulationView
ants = ArticulationView(prim_paths_expr="/World/Ants/.*/torso", name="ants_view")
my_world.scene.add(ants)

my_world.reset()
if args_cli.headless is True:
        enable_extension("omni.kit.livestream.native")
#for i in range(1000):
while simulation_app.is_running() and not simulation_app.is_exiting():
    print(ants.get_world_poses())
    my_world.step()
    if env.unwrapped.sim.is_stopped():
            break
    simulation_app.update()
simulation_app.close()

Anybody knows how to switch to the async version or slove it?

1 Like
20

@yiyiliu.liu We do not have a solution for an async version of get_assets_root_path() yet.
Have you installed a local Nucleus?
For now you can ignore the warning and it should not hang if you have access to a local Nucleus and Internet.