Simple Joint Control with Jupyter Notebook example - UR10 not moving

Omniverse Isaac Sim
1

I am trying to follow the simple joint control with jupyter notebook example after successfully setting-up the Isaac Omniverse simulator 2020.1 on a remote workstation using a Docker container, deploying the simple_joint_control package in my local machine and opening the jupyter notebook (also in my local machine).

Both applications (remote and local) report to connect to each other correctly. Remote:

2020-08-11 12:45:49.075 INFO  engine/alice/components/TcpPublisher.cpp@146: TCP publisher successfully accepted connection
2020-08-11 12:45:49.215 INFO  engine/alice/components/TcpSubscriber.cpp@170: Successfully connected to remote (10.0.0.53:55001). Will start receiving.

Local:

2020-08-11 13:45:49.074 INFO  engine/alice/components/TcpSubscriber.cpp@170: Successfully connected to remote (10.0.1.136:55000). Will start receiving.
2020-08-11 13:45:49.216 INFO  engine/alice/components/TcpPublisher.cpp@146: TCP publisher successfully accepted connection

However, while moving the sliders in the jupyter notebook the robot model remains still in the start position. The application graph on each side are as follows:

isaacsim.app.json

{
  "name": "isaacsim",
  "modules": [],
  "graph": {
    "nodes": [
      {
        "name": "interface",
        "components": [
          {
            "name": "ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "input",
            "type": "isaac::alice::TcpSubscriber"
          },
          {
            "name": "output",
            "type": "isaac::alice::TcpPublisher"
          },
          {
            "name": "time_sync",
            "type": "isaac::alice::TimeSynchronizer"
          }
        ]
      }
    ],
    "edges": []
  },
  "config": {
    "interface": {
      "input": {
        "port": 55001,
        "host": "10.0.0.53"
      },
      "output": {
        "port": 55000
      }
    }
  }
}

navsim_tcp.subgraph.json:

{
  "modules": [
    "navsim"
  ],
  "graph": {
    "nodes": [
      {
        "name": "interface",
        "components": [
          {
            "name": "ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "input",
            "type": "isaac::alice::TcpSubscriber"
          },
          {
            "name": "output",
            "type": "isaac::alice::TcpPublisher"
          },
          {
            "name": "time_sync",
            "type": "isaac::alice::TimeSynchronizer"
          }
        ]
      },
      {
        "name": "scenario_manager",
        "components": [
          {
            "name": "ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "scenario_manager",
            "type": "isaac::navsim::ScenarioManager"
          }
        ]
      }
    ],
    "edges": [
      {
        "source": "scenario_manager/scenario_manager/scenario_control",
        "target": "interface/input/scenario_control"
      },
      {
        "source": "scenario_manager/scenario_manager/robot",
        "target": "interface/input/scenario_actors"
      },
      {
        "source": "interface/output/scenario_reply",
        "target": "scenario_manager/scenario_manager/scenario_reply"
      }
    ]
  },
  "config": {
    "interface": {
      "input": {
        "port": 55000,
        "host": "10.0.1.136"
      },
      "output": {
        "port": 55001
      }
    },
    "scenario_manager": {
      "scenario_manager": {
        "ref_pose_name": "unity"
      }
    }
  }
}

The following errors are shown at the local side:

2020-08-11 13:45:48.911 ERROR engine/alice/components/Config.cpp@70: Could not find hook: command_generator/PyCodelet/joints
2020-08-11 13:45:48.911 ERROR engine/alice/components/Config.cpp@70: Could not find hook: command_generator/PyCodelet/limits

The remote side just shows some warnings that I can add later if they are believe to be relevant. Thank you in advance for any help provided.

2

The config error on local can be ignore.
Did you starting playing the simulation on Omniverse side? If there’s still issue after that, could you show a screenshot of the jupyter notebook with the slides?

3

Hi there,

Thank you for your answer. Yes, the simulation is running on the Omniverse side, I first start the Robot Engine Bridge and then hit play. Attached are the requested screenshots.

slides
slides1093×783 105 KB

4

Hi there,

Any updates?

5

I having similar issue here, UR10 no working

2020-09-28 00:59:39.324 ERROR engine/alice/components/Config.cpp@70: Could not find hook: command_generator/PyCodelet/joints
2020-09-28 00:59:39.324 ERROR engine/alice/components/Config.cpp@70: Could not find hook: command_generator/PyCodelet/limits
2020-09-28 00:59:39.325 INFO engine/alice/application.cpp@255: Starting application ‘simple_joint_control_ur10_sim’ (instance UUID: ‘d483c512-00e2-11eb-a216-077977028f07’) …
2020-09-28 00:59:39.325 DEBUG engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2020-09-28 00:59:39.325 DEBUG engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
2020-09-28 00:59:39.325 DEBUG engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2020-09-28 00:59:39.325 DEBUG engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
2020-09-28 00:59:39.325 INFO engine/alice/backend/asio_backend.cpp@33: Starting ASIO service
2020-09-28 00:59:39.329 ERROR packages/sight/WebsightServer.cpp@214: Failed to start Webserver on port 3000!
Make sure no other app is running and that the previous app has been closed properly
2020-09-28 00:59:39.329 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘websight/isaac.sight.AliceSight’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.331 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘lqr.kinematic_tree/KinematicTree’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.331 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-09-28 00:59:39.331 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘_check_operating_system/isaac.alice.CheckOperatingSystem’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.331 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-09-28 00:59:39.331 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-09-28 00:59:39.458 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-09-28 00:59:39.460 INFO engine/alice/components/TcpPublisher.cpp@75: Starting TCP publisher on port 55001
2020-09-28 00:59:39.487 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘simulation.interface/input’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.487 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘simulation.interface/output’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.487 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘simulation.interface/time_sync’ was not added to scheduler because no tick method is specified.
2020-09-28 00:59:39.487 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-09-28 00:59:39.487 INFO engine/alice/components/TcpSubscriber.cpp@140: TCP receiver trying to connect to remote (localhost:55000)…
2020-09-28 00:59:39.487 WARN packages/navsim/components/ScenarioManager.cpp@32: No scene specified. The default one loaded by navsim will be used.
2020-09-28 00:59:39.487 DEBUG engine/alice/backend/event_manager.cpp@39: Stopping node ‘simulation.scenario_manager’ because it reached status ‘SUCCESS’
2020-09-28 00:59:39.487 INFO engine/alice/components/TcpSubscriber.cpp@170: Successfully connected to remote (localhost:55000). Will start receiving.
2020-09-28 00:59:39.870 INFO engine/alice/components/TcpPublisher.cpp@146: TCP publisher successfully accepted connection
2020-09-28 00:59:49.325 INFO engine/alice/backend/allocator_backend.cpp@57: Optimized memory CPU allocator.
2020-09-28 00:59:49.325 INFO engine/alice/backend/allocator_backend.cpp@66: Optimized memory CUDA allocator.
[5639:5643:0928/005950.229737:ERROR:ssl_client_socket_impl.cc(963)] handshake failed; returned -1, SSL error code 1, net_error -200
Fontconfig error: Cannot load default config file: No such file: (null)
[5639:5643:0928/005950.893036:ERROR:ssl_client_socket_impl.cc(963)] handshake failed; returned -1, SSL error code 1, net_error -200
Fontconfig error: Cannot load default config file: No such file: (null)
[5639:5643:0928/005952.888507:ERROR:ssl_client_socket_impl.cc(963)] handshake failed; returned -1, SSL error code 1, net_error -200

6

From the notebook output, there’s N/A next to each join, this should be showing the current join angle state published from sim. N/A means the sdk app has not received join state message from sim yet. I’m not sure why this happens, if you are certain that play simulation is already started in sim and the sim to sdk TCP connection is established.
One thing you can try, is in the jupyter notebook class JointControl(Codelet) tick method, remove the logic that return early if no state message is received. This will allow the sdk app to still send join command to sim without receiving the join state from sim, then see if the arm moves in sim. This will test the sdk to sim TCP connection.