List of Suggested Future Features in Issac

Autonomous Machines Robotics - Isaac Isaac SDK
1

Hi all,

So my lab and I have been doing a bunch of work with Isaac and wanted to just compile a list of possible issues/suggestions for improvement that could be incorporated in future releases now that we have worked with it for a while. If there is anything here that is already fixed or there is an easily solution to please don’t hesitate to tell us :). We have split it up into several posts for ease of making additions and easy referencing.

In no particular order, here is our list:

2
  1. Isaac for ROS users documentation – So this is something which we thought would be nice to have on the documentation side. A lot of roboticists are used to ROS and there is a bunch of functionality in Isaac that is similar to what occurs in ROS. Having an easy way to know what is similar and what is different in terms of functionality and terminology would be great to have as a starting point for people who have known nothing but ROS. Something akin to this perhaps this page on Numpy for Matlab users NumPy for MATLAB users — NumPy v1.23 Manual. I think it would make it easier for people who are just getting started with Isaac and could increase adoption rates.
3
  1. Step by step robot building tutorial - So this point is probably just splitting hairs but it might be nice to have a more detailed step-by-step guide to creating complete robot systems using both home-made and pre-built codelets. There is a nice tutorial on how to get started with basic codelets with your ping-pong tutorials but it feels like it is a big leap to go from there to a full robotic system. We were happy for your examples but they didn’t help us gain much of an understanding of all that Isaac could offer. For that we had to go hurtling into source files as much as the existing documentation to get an understanding of what capabilities Isaac had and what was actually necessary to get a robot going. Also the fact that we kept running into pre-compiled libraries where we could only guess what was going on made it hard for us to try and implement things that were similar but slightly different to what you already had existing. I definitely understand there is only so much you can do there but any extra transparency through more detailed tutorials would definitely help others in future I think.
4
  1. Increased Debugging Toolset – One issue we had when using Isaac was that we found it harder to debug our code than what we expected. Some functionality that we would like/couldn’t find:
  • printing the current connected channels (type of protos being sent, something akin to rostopic list)
  • printing the output from a given codelet or along a given connected channel while code is running (something akin to rostopic echo). We would find such a thing preferrable to using the plotting visualisers in websight.
  • Adding the ability to see connection proto message types somehow within the App Graph in websight
1 Like
5
  1. Error message be thrown for incorrect proto types – This got us stuck for a long period of time at one point. Related to the above points about debugging tools, we would appreciate it if some sort of warning or error was thrown if you connect two sockets of different proto types. At the moment when we pass an incorrect type to a codelet, we get a “default” all-zeros output for any field where there was no data. Made worse was moments where some of the fields were the same between types of proto (FlatScanProto and RangeScanProto) and so some were distinct. In these instances we would get some “meaningful” numbers and some zeros. In short just throwing an error saying we got the type of message wrong in our connection would speed up the process of noticing and fixing these issues immensely.
1 Like
6
  1. Always following the last command given when instructions stop - This might be a simulator specific issue but thought I would raise it here in case it is more widespread. When we have the simulator running and our last command was something like “move forward” before we shut our bazel application/it closes unexpectedly, the robot will continue following the last given command and not stop moving forwards. In simulation this is annoying, if this transfers to the real robots it’s potentially dangerous. If there is some way to make sure that if you are getting no input you make no actions that would be beneficial.
7
  1. Desire for greater connectivity within Isaac Sim - So there always feels like there is a simulator of three parts whenever we use a robot within Isaac Sim. There is the part which is essentially just the unreal editor, there is the Isaac Sim config where we define robots in a config file which interacts with the unreal editor to define the robot model, Location of sensors etc (e.g. carter_sim_full.config.json) and the Isaac SDK config where we have the robot that actually interacts with code to control the robot using Isaac Gems etc. There have been moments where we have explicitly wanted to change one aspect of the robot such as the position of cameras and been annoyed by having to change the pose of the camera both in the Isaac Sim config and the Isaac SDK config manually rather than having a centralised location to do such things. Also the fact that they are operating off slightly different coordinate systems (at least in the examples provided) can prove a little vexing. It would also be really cool if we could do things like define the robot spawn position in the unreal editor rather than only in the config file. It looks like it has been a herculean task to get these systems to work together and I don’t personally know how this more synchronous approach can be performed but if it can be added it would be an extremely helpful feature for developers. Anything that reduces how many places we need to keep track of small changes the better :)
8

d20.hall

Thank you for your feedback.

  1. Isaac for ROS users documentation
    Could you check out the latest documentation and get back to us with your feedback?
    https://docs.nvidia.com/isaac/isaac/packages/ros_bridge/doc/ros_bridge.html

  2. Step by step robot building tutorial
    We have added another tutorial for Carter. Pls check it out.
    NVIDIA Carter — Isaac 2019.3 documentation

  3. Increased Debugging Toolset
    Thx will look into it.

  4. Error message be thrown for incorrect proto types
    Thx will look into it.

  5. Always following the last command given when instructions stop -

  6. Desire for greater connectivity within Isaac Sim
    Pls check out the latest Isaac Sim for navigation @ https://developer.nvidia.com/isaac/downloads
    https://docs.nvidia.com/isaac/isaac/doc/simulation/unity3d.html
    We are now using Unity 3D as the engine.
    Let me have your feedback reviewed.

Thanks!

1 Like
9

Hello d20.hall, thank you very much for the detailed feedback and for sharing you experience with Isaac SDK. I will plan to incorporate your requests into the next 2020.1 release.