IsaacROS Container Setup, Nitros feature image support

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hello Nvidia nvidia comunity,
I’m integrating the AM-Radio model into NVBlox-ROS to create 3D feature maps of the workspace of a robot arm. And i have a couple of questions.

  1. How can I make ROS deps/model installs persistent in the Isaac ROS container? I don’t have the base Dockerfile—what’s the recommended way to extend the base image (overlay Dockerfile/compose) so rosdep/pip/model weights survive rebuilds, or snapping the container?
    The docs reference a config file that is user accessible in the cli at ${ISAAC_ROS_WS}/.isaac-ros-cli/config.yaml
    do I need to create it and what am I able to do with it?

  2. The c-radio_v4-h model outputs 1536-dim features. I downsize to ~128 dims for lightweight visualization, but it’s still heavy to send over ROS. Is there a supported way to transport multi-channel float feature images with Nitros (or another compressed transport).

  3. What’s the min feature_array_num_elements recommended that would still yield results that look reasonable for debugging or would 128 for example even be enough to display a feature map of the workspace?

Also general thoughts about the Idea would be appreciated. In the end i want a decent approach to have a 3D feature map of the workspace, and this seemed to me to be a reasonable and scalable approach considering how much faster the AM-Radio model got and also considering the publishing of the RadSeg paper that would make it possible to have Semantic segmentation of said workspace.
Thank you for your help.

Best Reagards,
Tom

2

Hello @MossProphet,

Thanks for posting in the Isaac ROS forum!

I’ll answer your questions below.

  1. The Isaac ROS dev tooling mounts your workspace from the host into /workspaces/isaac_ros-dev inside the container. If you put your models (and other data) in a folder under this workspace on the host, they survive container rebuilds or deletion: you just re-enter the container and build/run as usual.

    If you also want apt/rosdep/pip installs themselves to be persistent, you need a custom image. The usual approach is using “docker commit” command to snapshot that container you’ve been running into a new image from the host and launch this image instead for your next time entering.

  2. For sending RADIO feature maps, you could try using regular ROS message and represent the feature map as a sensor_msgs::Image or PointCloud2. And alternative way could be using NITROS tensor type that treats the feature map as a single [H, W, C] float tensor since this tensor message type is intended for DNN features, then define a NITROS–ROS bridge node if needed.

  3. There’s no Isaac or NVBlox specific magic number here, it’s about how much information you want to preserve and what you’re doing with it in your implementation.