ROS2 communication between a docker container on AWS and a local Desktop environment

a.alrahmanalmrad · August 27, 2023, 5:15pm

Hello,
I have started recently a ROS2 project that operates remotly in a docker development container on AWS. My objective is to establish a direct bidirectional communication between ROS2 in this container and another ROS2 environment that was installed on a local Jetson. For additional context, ros2 will be used as part of isaac-sim ros2 bridge to communicate data from and to a robot placed in the simulation environment as part of a robotic digital twin project.
In the docker container:

Ubuntu version: 20.04

ROS2 version: foxy

ROS2 Environment variables:

ROS_IP =52.60.66.162

ROS_DOMAIN_ID =0

FASTRTPS_DEFAULT_PROFILES_FILE =~/.ros/fastdds.xml

“52.60.66.162” is the public IP address of the AWS EC2 instance that I am connected to on both the computer that is starting the remote docker container on AWS and the local Jetson
The fastdds.xml file was created in the <~/.ros> directory with the following content:

<?xml version="1.0" encoding="UTF-8" ?>

<license>Copyright (c) 2022, NVIDIA CORPORATION.  All rights reserved.
NVIDIA CORPORATION and its licensors retain all intellectual property
and proprietary rights in and to this software, related documentation
and any modifications thereto.  Any use, reproduction, disclosure or
distribution of this software and related documentation without an express
license agreement from NVIDIA CORPORATION is strictly prohibited.</license>

<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles" >
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>CustomUdpTransport</transport_id>
            <type>UDPv4</type>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="udp_transport_profile" is_default_profile="true">
        <rtps>
            <userTransports>
                <transport_id>CustomUdpTransport</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
        </rtps>
    </participant>
</profiles>

After pulling the following isaac-sim docker using the following command:

docker pull nvcr.io/nvidia/isaac-sim:2022.2.1

The docker container is created as follows in VSCode:

docker run --name isaac-ros-bridge-3 --entrypoint bash -it --gpus all -e "ACCEPT_EULA=Y" --network=host --ipc=host --pid=host     -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache/Kit:rw     -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw     -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw     -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw     -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw     -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw     -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw     -v ~/docker/isaac-sim/documents:/root/Documents:rw          -v /dev/shm:/dev/shm     public.ecr.aws/nvidia/isaac-sim:2022.2.1

On local Jetson:

Ubuntu version: 20.04

ROS2 version: foxy

ROS2 Environment variables:

ROS_IP = 172.17.0.1

ROS_DOMAIN_ID =0

FASTRTPS_DEFAULT_PROFILES_FILE =~/.ros/fastdds.xml

“172.17.0.1” is the IP address found on the local computer that I am supposed to communicate with using the terminal command. This IP address appears under docke0 inet (I used this local IP address since the docker container was created on top of the host network stack as you will later see in the command that created the container).

The fastdds.xml was the same as previously discussed and tried with and without it on the local Jetson.

When I start a simple talker node in the docker container using the following command:

ros2 run demo_nodes_py talker

and list the ros2 topic using: , I can see the topic /chatter listed.

But when I start a simple listener on the local machine using the following command:

ros2 run demo_nodes_py listener

I do not received the published messages and other way around gives the same behaviour (i.e. Talker started on local machine and listener started in the docker container on AWS).
For additional information: when I change the ROS_DOMAIN_ID to another number like 2 or 42… (while keeping it the same on both the container and the local Jetson environment) è I cannot see the /chatter topic listed anymore.

Also, when I start the talker on the Jetson I cannot see the /chatter topic listed in the AWS container

Any insights or guidance to help me solve this problem? Thanks in advance.

rchadha · August 29, 2023, 5:10pm

Hi @a.alrahmanalmrad , just to make sure I understand correctly: Are you looking to run ros2 on a remote and local machine? (Remote is AWS instance and local is Jetson). Have you been able to verify if this works as is with ros2?

Maybe try:

Run a ROS2 container on AWS instance (you can use official OSRF containers)
SSH into your instance via a Jetson, enter the container and verfiy ROS2 works as expected
Run a ROS2 container on the Jetson and see if you can get it to communicate now (might have to figure out some port forwarding stuff there, have pasted a few links below for reference)

This is probably going to be a ROS2 level issue because of the setup (not Isaac Sim).

Here are a few links which might help: Using Fast DDS Discovery Server as discovery protocol [community-contributed] — ROS 2 Documentation: Foxy documentation
Multi computer

Hope this helps, all the best!

a.alrahmanalmrad · August 29, 2023, 7:52pm

Thank you for your answer @rchadha . I am trying to establish ros2 communication between the container running in AWS and a local environment. Moreover, as i mentioned in my earlier post, I am planning to use this ROS2 communication to create a digital twin for a robot in Isaac-sim, therefore i have to use an isaac-sim container and not a ros2 container.

rchadha · August 29, 2023, 9:08pm

@a.alrahmanalmrad , yes using the Isaac Sim container is your end goal. However, I’m not sure if you will be able to have ros2 setup the way yo described (running docker container in cloud and trying to communicate with it via a local machine). That is the reason why I suggested using the OSRF container. Once that is working, we can figure out the Isaac Sim setup too (again: highlighting this probably is a ROS2 side limitation, independent of Isaac Sim)

a.alrahmanalmrad · August 29, 2023, 9:14pm

@rchadha I just realized that i cannot ping a local mahine connected to the AWS EC2 instance through SSH from within the instance. Do you think i should make this work first before going ahead and establishing any type of connection?

rchadha · August 29, 2023, 9:29pm

yes, that’s what I have been trying to say. Communication issues between local/remote instance have to be sorted before getting into ROS2 issues.

a.alrahmanalmrad · August 29, 2023, 9:34pm

ok, so right now i have a docker container with its own IP under the public IP address of the AWS EC2 instance and I have a local Jetson with its own IP under the public IP address of my company. What do you suggest to do so that i connect the Jetson and the docker container so that i can proceed to solve ROS2 issues? I realize that this is a pure networking question but i hope you can provide some guidance. Thx

rchadha · August 29, 2023, 9:55pm

@a.alrahmanalmrad , please refer to my first answer on this post. I have mentioned few things you can try out there

a.alrahmanalmrad · September 1, 2023, 12:08am

@rchadha i spent the last couple of days trying to establish a pure ro2 connection between a talker node running in the AWS docker container and a listener node running on the local Jetson but nothing worked. I believe that i am facing trouble with ports forwarding or i am not setting fastdds parameters correctly. It seems that what i am trying to achieve is not a standard or usual practice and i am hoping that you can suggest additional solutions for this situation. Thank you