Issues with Franka Robot Not Moving in Isaac Sim "Hello_world" Tutorial

Omniverse Isaac Sim
1

I’m currently working through the Isaac Sim tutorials and have encountered an issue with the “Hello_world” app, specifically in section 5.1.4. After loading the app, the Franka robot model gets stuck in its initial position and does not respond when I try to move the target. I have followed all the steps in the tutorial carefully, but the problem persists.

Additionally, I’ve encountered several warnings and errors in the console logs, which I’m unsure how to resolve. Here’s a summary of the log messages:

  • Connection Status: Successfully connected to the Omniverse server on localhost.
  • Warnings about UsdStage Reference Counts: Indicating potential memory management issues.
  • URDF Import Warnings: Specifically mentioning that panda_link8 lacks colliders and inertia, and a small isotropic inertia matrix was assigned.
  • Type Mismatch Error: For a damping property, expecting ‘float’ but got ‘double’.
  • PhysX Plugin Warnings: Issues parsing tendon axis due to topology constraints and more than 4 velocity iterations being added to a TGS scene.
  • Dynamic Control and PhysX Tensors Plugin Errors: Failed to find articulation at ‘/panda’, indicating issues with articulation initialization.
  • AttributeError in Isaac Sim Extension: Related to an object not having the ‘is_homogeneous’ attribute, possibly during scene reset or robot initialization.

I’ve included these details hoping someone might point out what I’m missing or doing wrong. Has anyone experienced similar issues or can offer advice on how to troubleshoot these errors? I’m particularly stuck on why the Franka robot does not move and how to address the console errors and warnings.

Any help or guidance would be greatly appreciated. Thank you in advance for your support!
[307.022s] [ext: omni.isaac.sim-2023.1.1] startup
2024-02-04 23:56:01 [319,239ms] [Error] [asyncio] Task exception was never retrieved
future: <Task finished name=‘Task-1328’ coro=<BaseSampleExtension._on_load_world.._on_load_world_async() done, defined at /home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample_extension.py:152> exception=Exception(‘Cannot add the object target to the scene since its name is not unique’)>
Traceback (most recent call last):
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample_extension.py”, line 153, in _on_load_world_async
await self._sample.load_world_async()
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample.py”, line 47, in load_world_async
await self._world.reset_async()
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py”, line 518, in reset_async
await self.reset_async_set_up_scene()
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py”, line 444, in reset_async_set_up_scene
task.set_up_scene(self.scene)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/tasks/follow_target.py”, line 76, in set_up_scene
self.set_params(
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/tasks/follow_target.py”, line 116, in set_params
self._target = self.scene.add(
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py”, line 100, in add
raise Exception(“Cannot add the object {} to the scene since its name is not unique”.format(obj.name))
Exception: Cannot add the object target to the scene since its name is not unique
2024-02-04 23:56:10 [328,269ms] [Warning] [omni.usd] Unexpected reference count of 2 for UsdStage ‘anon:0x7f2ec800b540:World3.usd’ while being closed in UsdContext (this may indicate it is still resident in memory).
2024-02-04 23:56:10 [328,910ms] [Warning] [omni.isaac.core.simulation_context.simulation_context] A new stage was opened, World or Simulation Object are invalidated and you would need to initialize them again before using them.
2024-02-04 23:56:24 [342,359ms] [Warning] [omni.usd] Unexpected reference count of 2 for UsdStage ‘anon:0x7f3cf80008d0:World4.usd’ while being closed in UsdContext (this may indicate it is still resident in memory).
2024-02-04 23:56:24 [342,613ms] [Warning] [carb] Client omni.physx.plugin has acquired [carb::settings::ISettings v1.0] 100 times. Consider accessing this interface with carb::getCachedInterface() (Performance warning)
2024-02-04 23:56:25 [343,348ms] [Warning] [omni.importer.urdf] Link panda_link8 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2024-02-04 23:56:25 [343,452ms] [Warning] [omni.usd] Coding Error: in _SetValueImpl at line 6189 of /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp – Type mismatch for </panda/panda_hand/panda_finger_joint1.physxTendon:panda_finger_joint2:damping>: expected ‘float’, got ‘double’

2024-02-04 23:56:25 [343,775ms] [Warning] [carb] Client omni.stageupdate.plugin has acquired [omni::hydra::IOmniHydra v2.0] 100 times. Consider accessing this interface with carb::getCachedInterface() (Performance warning)
2024-02-04 23:56:25 [343,950ms] [Warning] [omni.physx.plugin] Could not parse tendon axis at /panda/panda_hand/panda_finger_joint2 due to a topology issue: Refer to the topology constraints in the USD schema doc for PhysxTendonAxisAPI.
2024-02-04 23:56:25 [343,950ms] [Warning] [omni.physx.plugin] The fixed tendon axis at /panda/panda_hand/panda_finger_joint2 was not parsed due to a topology issue: Refer to the topology constraints in the USD schema doc for PhysxTendonAxisAPI.
2024-02-04 23:56:25 [344,021ms] [Warning] [omni.isaac.dynamic_control.plugin] Failed to find articulation at ‘/panda’
2024-02-04 23:56:25 [344,021ms] [Error] [omni.isaac.dynamic_control.plugin] DcGetArticulationRootBody: Invalid or expired articulation handle
2024-02-04 23:56:25 [344,021ms] [Warning] [omni.physx.tensors.plugin] Failed to find articulation at ‘/panda’
2024-02-04 23:56:25 [344,021ms] [Error] [omni.physx.tensors.plugin] Pattern ‘/panda’ did not match any articulations

2024-02-04 23:56:25 [344,022ms] [Error] [asyncio] Task exception was never retrieved
future: <Task finished name=‘Task-1382’ coro=<BaseSampleExtension._on_load_world.._on_load_world_async() done, defined at /home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample_extension.py:152> exception=AttributeError(“‘NoneType’ object has no attribute ‘is_homogeneous’”)>
Traceback (most recent call last):
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample_extension.py”, line 153, in _on_load_world_async
await self._sample.load_world_async()
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.examples/omni/isaac/examples/base_sample/base_sample.py”, line 47, in load_world_async
await self._world.reset_async()
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py”, line 520, in reset_async
await self.reset_async_no_set_up_scene(soft=soft)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py”, line 481, in reset_async_no_set_up_scene
self._scene._finalize(self.physics_sim_view)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py”, line 322, in _finalize
robot.initialize(physics_sim_view)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.franka/omni/isaac/franka/franka.py”, line 118, in initialize
super().initialize(physics_sim_view)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py”, line 294, in initialize
self._articulation_view.initialize(physics_sim_view=physics_sim_view)
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py”, line 349, in initialize
assert self._physics_view.is_homogeneous
File “/home/pkoprov/.local/share/ov/pkg/isaac_sim-2023.1.1/extsPhysics/omni.physics.tensors-105.1.12-5.1/omni/physics/tensors/impl/api.py”, line 170, in is_homogeneous
return self._backend.is_homogeneous
AttributeError: ‘NoneType’ object has no attribute ‘is_homogeneous’

Screenshot from 2024-02-04 19-00-39
Screenshot from 2024-02-04 19-00-39658×589 288 KB

2

@pkoprov i experienced the same situation of robot not moving by following all the way through 5.1.3 and the message regarding panda_link8. but, once you are loading the franka URDF through python via editing the hello_world.py (5.1.4) you will have a target prim to move around in your scene with franka following.

so not sure if the instruction prior to 5.1.4 was just to show how to load and use the extension (since there wasn’t any steps that set set up the target prim/task)

3

Yes, that is the problem. Loading the robot through Python API is not working. Actually, loading is working but the task is not being performed. I may take a look to find the issue later.

4

if you are inclined, feel free upload your edited hello_world.py and i can try to repro on my end as is.

5

Hello,

My approach is:

  1. First import a robot or use the example, but make sure that you have the right articulation root, which is \panda not \panda\panda_link0 by changing the articulation root, please refer to Tut 2.4 on how to set up Articulation Root.
  2. Now you should have the “appropriate” .usd file
  3. Load the .usd file; Do not try to import urdf through script because it Isaac Sim will get the wrong articulation root (I do not know why)
  4. Watch Avi’s tutorial: https://www.youtube.com/watch?v=87q_ATDhrVY

BUT, DO NOT TRY TO LOAD YOUR ROBOT FROM SCRIPT (for 2023.1.1 release); I mean at the extension step please use the configuration template type.
This is my case, I do not know the articulation problem belong to my computer or it is a “feature” in this Isaac Sim release

7

Here it is:

from omni.isaac.core.utils.extensions import get_extension_path_from_name
from omni.importer.urdf import _urdf
from omni.isaac.franka.controllers import RMPFlowController
from omni.isaac.franka.tasks import FollowTarget
import omni.kit.commands
import omni.usd


class HelloWorld(BaseSample):
    def __init__(self) -> None:
        super().__init__()
        return


    def setup_scene(self):
        world = self.get_world()
        world.scene.add_default_ground_plane()
        # Acquire the URDF extension interface
        urdf_interface = _urdf.acquire_urdf_interface()
        # Set the settings in the import config
        import_config = _urdf.ImportConfig()
        import_config.merge_fixed_joints = False
        import_config.convex_decomp = False
        import_config.fix_base = True
        import_config.make_default_prim = True
        import_config.self_collision = False
        import_config.create_physics_scene = True
        import_config.import_inertia_tensor = False
        import_config.default_drive_strength = 1047.19751
        import_config.default_position_drive_damping = 52.35988
        import_config.default_drive_type = _urdf.UrdfJointTargetType.JOINT_DRIVE_POSITION
        import_config.distance_scale = 1
        import_config.density = 0.0

        # Get the urdf file path
        extension_path = get_extension_path_from_name("omni.importer.urdf")
        root_path = extension_path + "/data/urdf/robots/franka_description/robots"
        file_name = "panda_arm_hand.urdf"

        # Finally import the robot
        result, prim_path = omni.kit.commands.execute( "URDFParseAndImportFile", urdf_path="{}/{}".format(root_path, file_name),
                                                      import_config=import_config,)
        my_task = FollowTarget(name="follow_target_task", franka_prim_path=prim_path,
                               franka_robot_name="fancy_franka", target_name="target")
        world.add_task(my_task)
        return

    async def setup_post_load(self):
        self._world = self.get_world()
        self._franka = self._world.scene.get_object("fancy_franka")
        self._controller = RMPFlowController(name="target_follower_controller", robot_articulation=self._franka)
        self._world.add_physics_callback("sim_step", callback_fn=self.physics_step)
        await self._world.play_async()
        return

    async def setup_post_reset(self):
        self._controller.reset()
        await self._world.play_async()
        return

    def physics_step(self, step_size):
        world = self.get_world()
        observations = world.get_observations()
        actions = self._controller.forward(
            target_end_effector_position=observations["target"]["position"],
            target_end_effector_orientation=observations["target"]["orientation"],
        )
        self._franka.apply_action(actions)
        return
9

Maybe I was not very clear, but I did not have any problems wit anything from 5.1.1 to 5.1.4. Only when I am trying to run Python script, I have this problem. I will try to find what is causing the trouble, because the objects do spawn, but don’t move.