OK we got further than some others who tried this example in Omniverse.
Following the documentation here:
The summary is that everything seems to work as expected, with the STR transporter recognizing the dolly, approaching it, showing the bounding box, creating a goal to go under the dolly, and then moving towards the goal.
However, viewing this in sight, it appears that as the STR transporter robot is getting closer to the dolly, the goal itself is not stationary and instead also moves towards the STR. So it reaches the success status of being at the goal position before the STR is under the dolly.
Then the lifter is activated, and the spherical model shape of the robot changes as if the dolly was in fact docked, and the STR moves to the destination.
Here are some screenshots:
Omniverse view of STR approaching dolly, everything looks ok:
Sight view of STR approaching dolly, everything looks ok, blue robot shape in the planner view is moving towards the green goal:
Now the new goal is generated with the goal position initially under the dolly, centered between the 4 wheel obstacles. But almost immediately it seems that the goal position is moving towards the STR just as the STR is approaching the goal, and this shouldn’t happen
Now the STR has reached its goal and changed its shape, but the dolly is not on the STR
It looks like we’re very close in seeing this example of cooperation between Isaac SDK and Omniverse Isaac Sim working.
We do understand that this example was originally developed in Unity3D and all the training was probably done in Unity3D. And it’s amazing that Nvidia managed to create a simulation engine that has much superior rendering and physics and yet compatible enough that the simulation almost works the same.
So, 3 questions:
Is the assumption correct that this example was fully functional with Unity3D ? The only system we have that still runs Unity3D doesn’t have enough VRAM to run this example.
was this ever fully functional with Omniverse Isaac Sim ?
we didn’t modify any source code, and in fact re-loaded the Omniverse Isaac SDK assets so everything is as it was distributed. Any idea why the example isn’t working now ?
Turning anti-aliasing on makes the image more pleasing without change in behavior. Turning reflections and translucency off increases frame rate but that also doesn’t change the behavior.
The results are repeatable, and moving things around for a slightly different initial position also doesn’t change the behavior.
Looks like the RTX2060 is not running out of resources:
We tried upgrading to RTX3060 but that causes TensorRT errors, crashing the Isaac SDK process.
Thanks for any insight