ESS DNN Stereo Disparity on a Jetson Xavier NX

Autonomous Machines Robotics - Isaac Isaac ROS
, ,
1

Hi there, I’m hoping someone can help me out or point me in the right direction for implementing the Stereo Disparity neural network from Isaac ROS. It is my current understanding that it should be possible to use the model on my own data. Ultimately I would like to use it for real time inference within a Python program.

I’ve tried following the instructions no the GitHub page, however on two different computers the commands have failed at different points. Am I wrong in thinking this is something that is available for immediate use with applications? Am I right in thinking I can use this model in my own Python application?

Thanks in advance!

2

Yes, Isaac ROS DNN Stereo Disparity is available for immediate use with your ROS2 Humble application. Could you describe what the commands were that failed and more about the error messages you may have seen?

3

Hi hemals, thanks for your response.

Running Ubuntu 20.04 and Cuda 11.6.124, following the instructions on this page GitHub - NVIDIA-ISAAC-ROS/isaac_ros_dnn_stereo_disparity: Deep learned stereo disparity estimation results in the following error during the " * Launch the Docker container using the run_dev.sh script:" step.

shane@shane-TensorBook:~/workspaces/issac_ros-dev/src/isaac_ros_common$ ./scripts/run_dev.sh
isaac_ros_dev not specified, assuming /home/shane/workspaces/issac_ros-dev/src/isaac_ros_common
~/workspaces/issac_ros-dev/src/isaac_ros_common ~/workspaces/issac_ros-dev/src/isaac_ros_common
Building x86_64.humble.nav2.user base as image: isaac_ros_dev-x86_64 using key x86_64.humble.nav2.user
Using base image name not specified, using ''
Using docker context dir not specified, using Dockerfile directory
Resolved the following Dockerfiles for target image: x86_64.humble.nav2.user
/home/shane/workspaces/issac_ros-dev/src/isaac_ros_common/scripts/../docker/Dockerfile.user
/home/shane/workspaces/issac_ros-dev/src/isaac_ros_common/scripts/../docker/Dockerfile.x86_64.humble.nav2
Building /home/shane/workspaces/issac_ros-dev/src/isaac_ros_common/scripts/../docker/Dockerfile.x86_64.humble.nav2 as image: x86_64-humble-nav2-image with base: 
Sending build context to Docker daemon  82.43kB
Step 1/1 : FROM nvcr.io/nvidia/isaac/ros:x86_64-humble-nav2_53c4c553a22864a0675e9b05bb7cf19b
 ---> 1bb9ed822f08
[Warning] One or more build-args [USERNAME USER_GID USER_UID] were not consumed
Successfully built 1bb9ed822f08
Successfully tagged x86_64-humble-nav2-image:latest
Building /home/shane/workspaces/issac_ros-dev/src/isaac_ros_common/scripts/../docker/Dockerfile.user as image: isaac_ros_dev-x86_64 with base: x86_64-humble-nav2-image
Sending build context to Docker daemon  82.43kB
Step 1/17 : ARG BASE_IMAGE
Step 2/17 : FROM ${BASE_IMAGE}
 ---> 1bb9ed822f08
Step 3/17 : ARG USERNAME=admin
 ---> Using cache
 ---> 095691121ed0
Step 4/17 : ARG USER_UID=1000
 ---> Using cache
 ---> 50833042f84e
Step 5/17 : ARG USER_GID=1000
 ---> Using cache
 ---> 24474df99fee
Step 6/17 : RUN apt-get update && apt-get install -y         sudo && rm -rf /var/lib/apt/lists/* && apt-get clean
 ---> Running in 20a50cd3eecc
failed to create shim task: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v2.task/moby/20a50cd3eecc9650633bbcdc842f4a39f887c75161f6c532d035fafc284da501/log.json: no such file or directory): exec: "nvidia-container-runtime": executable file not found in $PATH: unknown
Failed to build base image: isaac_ros_dev-x86_64, aborting.
~/workspaces/issac_ros-dev/src/isaac_ros_common
4

Can confirm that you made the dev env setup steps mentioned in our quickstar section of our readmes has been completed?

5

Hi there, I actually realized that I had not installed ROS yet - that was the issue!

6

The run_dev.sh script should work without ROS installed locally, but after you follow the dev env setup steps. ROS is part of the docker container we build in isaac_ros_common.

7

Oh ok! Might have been something else then.
How would I be best able to use this model in my own Python program with my own stereo rectified image pairs?
I have it working with my own images but would like to make it work with live video input.

8

The Isaac ROS DNN Stereo Disparity package’s ESSDisparityNode expects the rectified image pairs and corresponding camera information published to a set of four ROS topics as specified here.

Which kind of camera are you trying to use here, and what specifically would you like your Python program to do?

If you have an Argus-compatible camera in your system, then you can use our existing Isaac ROS ArgusStereoNode to publish to those topics, as shown in this tutorial.

9

Thank you for pointing me to that information - I’m new to ROS but can already see how the package would fit in to a pipeline.

My Python program is intended to receive stereo image pairs via OpenCV, and I would like to run both face detection and the ESS DNN Disparity model to calculate the distances to the faces.

Is there a way to implement the ESS DNN model within a Python program? I’m using an open-source library for face detection (InsightFace) and would love if there was a way to implement the disparity model within my Python program as well.

10

You can configure a launch file with your graph with the Isaac ROS DNN Stereo Depth Disparity packages and your Python code as a separate node that listens to the output topics.