Dropping frames in the new environment

xmaster · September 10, 2019, 7:01am

Hi,

I was using the Isaac Sim and SDK (Carter_sim with joystick control application) to collect data in our own environment and ran into the ‘dropping frame’ problem. Please have a look at the images in this folder for an example:
https://drive.google.com/drive/folders/1vP51C9gNJDG9u-16ZoisuRsnlVqYVecL?usp=sharing

There is an obvious dropping frame problem between these images, especially between 916.png and 919.png. Actually, this problem happened frequently in the data collection process, making the video quite not smooth. This problem only happened in the new version of Isaac Sim and Isaac SDK (2019.2) and I did not come across it in the old version.

When the problem happened, the terminal printed:

2019-09-10 16:23:44.560 DEBUG engine/alice/backend/node_backend.cpp@342: Stopping node ‘scan_localization’
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘scan_localization/robot_viewer’ …
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘scan_localization/robot_viewer’ DONE
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘scan_localization/isaac.navigation.ParticleFilterLocalization’ …
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘scan_localization/isaac.navigation.ParticleFilterLocalization’ DONE
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘scan_localization/flatscan_viewer’ …
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘scan_localization/flatscan_viewer’ DONE
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/node_backend.cpp@342: Stopping node ‘global_plan’
2019-09-10 16:23:44.560 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘global_plan/isaac.planner.GlobalPlanner’ …
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘global_plan/isaac.planner.GlobalPlanner’ DONE
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘global_plan/isaac.navigation.OccupancyToBinaryMap’ …
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘global_plan/isaac.navigation.OccupancyToBinaryMap’ DONE
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@97: Stopping codelet ‘global_plan/isaac.navigation.BinaryToDistanceMap’ …
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@107: Stopping codelet ‘global_plan/isaac.navigation.BinaryToDistanceMap’ DONE
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/node_backend.cpp@317: Starting node ‘global_localization’
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@61: Starting codelet ‘global_localization/grid_search_localizer’ …
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@73: Starting codelet ‘global_localization/grid_search_localizer’ DONE
2019-09-10 16:23:44.561 DEBUG engine/alice/backend/codelet_backend.cpp@291: Starting job for codelet ‘global_localization/grid_search_localizer’
2019-09-10 16:23:44.652 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.
2019-09-10 16:23:44.753 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.
2019-09-10 16:23:44.806 INFO packages/navigation/GridSearchLocalizer.cpp@175: Global localization finished in 0.229946 seconds
2019-09-10 16:23:44.851 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.
2019-09-10 16:23:44.951 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.
2019-09-10 16:23:45.052 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.
2019-09-10 16:23:45.170 WARNING packages/planner/DifferentialBaseLqrPlanner.cpp@355: LQR: Did not finish to converge

and sometimes only printed:

2019-09-10 16:25:35.054 ERROR packages/planner/DifferentialBaseLqrPlanner.cpp@178: No valid path for the robot! Emergency stop.

Could you please help me out with this issue? Thank you.

Kind regards,
Xmaster

TravisM · September 10, 2019, 5:13pm

I have experienced an issue with the Scan_Localization node inappropriately stopping frequently experienced in the stock (as it comes with fresh Isaac 2019.2, no changes made by me) carter_sim, only since 2019.2 as well. This causes the simulated robot to stop, drift, or overcorrect depending on the circumstances during the stoppage of the scan_localization node.

I described this issue in this and the Sim forums:

but have not gotten any feedback to confirm whether or not this is simply an issue with my environment (once again, I am not running my own code in this scenario - simply the carter_sim that came with Isaac 2019.2, in the latest version of Isaac Sim documented in the 2019.2 docs)

Good luck running this down, I am curious as to the cause of these issues, if they are indeed related.

xmaster · October 31, 2019, 1:02am

This problem gets worse when the new environment has a large scale. Any help?

shrinv · November 9, 2019, 10:03pm

Sorry about the delay in response XMaster.
Can you tell us more about your set up?
What’s your workstation configuration? What GPU are you using? Driver version etc.
And, Isaac Sim - are you using the one based on Unity? Or UE4?

xmaster · November 13, 2019, 1:29am

The Isaac Sim I am using is based on UE4.

The point is there was no such a frame-dropping problem with the old version of Isaac Sim.

Thank you.

xmaster