Long building time with docker and ISAAC ROS 4.5

Robotics & Edge Computing Robotics - Isaac Isaac ROS
,
1

Hello,

I am having a hard time understanding how the Docker configuration works in the new 4.5 ISAAC ROS version.

I’m using a Jetson Orin Nano board and I recently updated from Jetpack 6 to Jetpack 7.2. As I also wanted to use a newer ISAAC ROS version, I updated from 3.2 to 4.5.

However since the update I’m facing very long building times : basically every time I make the slightest change to any Dockerfile, it rebuilds the whole stack (and that takes around 30 - 40 minutes).

For example, let’s say I need to use a ZED X camera so I need to add the custom zed image to the configuration file ~/.config/isaac-ros-cli/config.yaml (exactly as described in the tutorial).

In addition to this, I also need to install other packages (for example nano and kmod) so I would have to create another Dockerfile into /etc/isaac-ros-cli/docker/Dockerfile.custom with the following content :

ARG BASE_IMAGE
FROM ${BASE_IMAGE}

RUN apt-get update && apt-get install -y kmod nano

I would modify my configuration file to this :

docker:
  image:
    additional_image_keys:
      - zed
      - custom

And run the build command :

isaac-ros activate --build-local

Now if I change something in Dockerfile.custom (for example if I add a package to the install command) then the whole ZED image is also fully rebuilt when running isaac-ros activate --build-local.

I think I’m missing something, because in the 3.2 version of ISAAC ROS the behavior was different (only the modified image was rebuilt). What am I doing wrong ?

Thank you for your time,
BR

2

Hello @paul.meis,

Welcome to the Isaac ROS forum, and thanks for the post!

When you’re using the docker configuration for different layer, Zed and custom are not part of the predefined image_key_order, so they are sorted alphabetically. As a result, the actual order is isaac_ros → custom → zed, even though zed appears first in additional_image_keys.
Changing Dockerfile.custom therefore changes the base used by Dockerfile.zed, invalidating the ZED layer.

As a workaround, rename the custom layer to something like Dockerfile.zz_custom so that it sorts after zed:

additional_image_keys:

  • zed
  • zz_custom

Run one full rebuild and verify that the reported image key ends in zed.zz_custom.
Subsequent custom-layer changes should reuse the cached ZED stage.

Also, Jetson Orin Nano with JetPack 7.2 is not a supported Isaac ROS 4.5 combination. Please recreate the environment with Isaac ROS 4.6, which officially supports Jetson Orin and JetPack 7.2.

3

Hello @vchuang,

Ok I didn’t know that ! After renaming the Dockerfile keys it works as intended now.
And yes I updated to Isaac ROS 4.6 in the meantime, so it should be good.

Thank you very much for your support ! Maybe it would be nice to add this to the documentation ;)

BR