I want to dynamically load global maps. For example, I have map of building A, B, C.
When the robot goes from A to B, the map can be dynamically updated.
A better version is that, I have a set of tiles in map. I want to load a subset of tiles during the movement of the robot.
Currently, Isaac SDK only support to read a static map with xxx.graph.json, xxx.config.json and xxx.png.
Would it be possible to opensource the Map code so that we can write our own map module.
Below are more details:
For the map, I want to create a map interface (Abstraction Layer). I can dynamically load tiles of maps when the robot moves. I want to know how the current map interactive with other modules in isaac. Currently, map is with Image and lattice. But there are many hidden hard coded links. The hidden links to the map are hard to track.
For the planner, I want to implement a flexible motion planner. But current code has many internal links. For example, isaac::navigation::GlobalPlanner. In the API, it says only need to receive goals and plan a path from current position to the target. Where does the current position come? Where does the map come from? Also there is a parameter: “disable_automatic_start”. When it is set to true, how do I activate it when I need a call?
Even I want to write a very simple A-STAR on the given map and pass it to navigation. It is not easy to pass information properly. isaac.perception.FiducialAsGoal gives a good example of how to write our own planner. But it is not open sourced. It would be better to have one example like isaac.perception.FiducialAsGoal. It outputs a plan to track
Thanks for your patience, @qazmichaelgw. We’re working on looping in some of the ISAAC team to help us get you more useful answers.
For visualizing messages between components, while not straightforward yet, you could try using the Recorder component (//engine/alice/components/Recorder.hpp/cpp) and tap the messages you want to capture to write as JSON to a Cask file (//engine/gems/cask/cask.hpp/cpp) which is really an LMDB database file. It isn’t ideal for sure, but it could work to help you see what the messaging looks like.
isaac::map::Map is a codelet that manages a set of layers, including isaac::map::OccupancyGridMapLayer which reads in the fixed PNG, and isaac::map::NavigationMap layer. Do you intend to replace this component and then expect the rest of the stack (localizer, planner, etc.) to behave as intended?Conceptually, there should be no hidden coupling beyond message passing in the graph. A Map interface abstraction should not be necessary or useful here since you should be able to swap out the isaac::map::Map component with something else that emits Image and Lattice protos.
However, I think I see what you mean by hidden links: one node component is able to get directly reference to another in the same node without needing to establish a graph edge. In packages/flatscan_localization/apps/localization.subgraph.json, the GridSearchLocalizer is getting direct access to isaac::navigation::NavigationMap by class name, which means you would not be able to swap out isaac::map::Map with something else entirely. This coupling may make it very difficult to use the rest of the navigation stack with a custom map component.
There are also plenty of assumptions that the map does not change during the lifecycle of these components. The recommended way for now is to restart the nodes with your new map, but I understand that may not be feasible for you.
isaac::map::OccupancyGridMapLayer which you wanted to replace)disable_automatic_start is an annotation for the graph to not start the node with the rest of the application, but only manually through a call to the app backend scheduler to start it. I am not sure yet how you make that call to start the node, but I will investigate.
FiducialAsGoal does not actually use a map but just emits the pose of the fiducial projected to the floor in the frame of the robot continuously.It would be very helpful to nail down a distilled use case that is as simple as possible but would make it an easy substitution for your method. Basic Fiducial Navigation Sketch.docx (296.0 KB) is a one-pager description of the most basic visual mapping case I could think of, and a slightly more interesting use case Fiducial_GPS Navigation Sketch.docx (301.2 KB) . Could you comment on whether this is similar to your use case or, if we could implement these use cases on top of ISAAC, you would have everything you think you would need?
Even with source, I’m not convinced it would be straightforward to substitute components in the navigation stack the way I think you want to yet. I’d like to determine what those gaps are with this exercise above and find some workarounds for you in the short term to unblock you though.
Hello hemals,
Thanks a lot for your summary and detailed reply. It is correct, those hidden links forbid the flexibility of changing components. The Use Case Sketch of Fiducial GPS can be treated as one good example. It is similar to my use case. To add on, there would be some local map when no Fiducials are visible from local sensing. To simplify the case, 2d lattice would be enough. In my future implementation the map maybe voxel based or even more complicated self-defined data structure. This use case can shed light on how to integrate a dynamic map rather than one PNG static map with lattice.
For the view of the topic, it might not be a good way. The use case is that the APP Graphs shows the connection of all components. But some capn protocals may have no real tx or rx messages. It is hard for me to check whether a tx or rx messages is actually there.