Arducam IMX219 isaac_ros_argus_camera support

[sascha8a]

Hi,

I am currently trying to get the arducam IMX219 working with isaac_ros_argus_camera. I am aware, that there is no official support for the camera, nonetheless I would like to get it working.

If need be, I can modify the device tree and IMX219 camera driver.

System Configuration
Jetpack 6.1
ROS humble with isaac ros
Working IMX219 (using gstreamer / gscam)

Errors and steps to reproduce
I am starting the argus_camera node with the following command:

ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_mono.launch.py

This produced the following output of the argus node:

[component_container_mt-1] [INFO] [1740332990.029276721] [argus_mono_container]: Load Library: /opt/ros/humble/lib/libmono_node.so
[component_container_mt-1] [INFO] [1740332990.129677835] [argus_mono_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusMonoNode>
[component_container_mt-1] [INFO] [1740332990.129820176] [argus_mono_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusMonoNode>
[component_container_mt-1] [INFO] [1740332990.148093889] [argus_mono]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1740332990.148866140] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1740332990.150955816] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_gxf_helpers.so
[component_container_mt-1] [INFO] [1740332990.159326357] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sight.so
[component_container_mt-1] [INFO] [1740332990.168038670] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_atlas.so
[component_container_mt-1] 2025-02-23 17:49:50.186 WARN  gxf/std/program.cpp@538: No GXF scheduler specified.
[component_container_mt-1] [INFO] [1740332990.188947517] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1740332990.193735369] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_argus.so
[component_container_mt-1] 2025-02-23 17:49:50.207 WARN  gxf/std/program.cpp@538: No GXF scheduler specified.
[component_container_mt-1] [INFO] [1740332990.207740577] [argus_mono]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1740332990.213528305] [argus_mono]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1740332990.213899038] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1740332990.217757673] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_message_compositor.so
[component_container_mt-1] [INFO] [1740332990.218573990] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1740332990.219562890] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1740332990.230828127] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_tensorops.so
[component_container_mt-1] [INFO] [1740332990.236926106] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_rectify.so
[component_container_mt-1] [INFO] [1740332990.242117077] [argus_mono]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_timestamp_correlator.so
[component_container_mt-1] [INFO] [1740332990.252555756] [argus_mono]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1740332990.256225456] [argus_mono]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1740332990.297317525] [argus_mono]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1740332990.303748796] [argus_mono]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/argus_mono' in container '/argus_mono_container'
[component_container_mt-1] [INFO] [1740332991.304455458] [argus_mono]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1740332991.304616904] [argus_mono]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1740332991.304672618] [argus_mono]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1740332991.304708204] [argus_mono]: [NitrosPublisher] Use only the compatible publisher: topic_name="/left/image_raw", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1740332991.304737869] [argus_mono]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1740332991.304763246] [argus_mono]: [NitrosPublisher] Use only the compatible publisher: topic_name="/left/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1740332991.305279776] [argus_mono]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1740332991.322330725] [argus_mono]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/SAEKBTRSPG/SAEKBTRSPG.yaml"
[component_container_mt-1] [INFO] [1740332991.322453801] [argus_mono]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1740332991.330003673] [argus_mono]: [ArgusCameraNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1740332991.330231841] [argus_mono]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1740332991.464236690] [argus_mono]: [NitrosNode] Node was started
[ERROR] [component_container_mt-1]: process has died [pid 16018, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args --log-level info --ros-args -r __node:=argus_mono_container -r __ns:=/'].

And the following in the nvargus-daemon:

=== NVIDIA Libargus Camera Service (0.99.33)=== Listening for connections...=== component_container_mt[16111]: Connection established (FFFF846FB840)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
=== component_container_mt[16111]: CameraProvider initialized (0xffff7cb41320)=== component_container_mt[16111]: Connection closed (FFFF846FB840)=== component_container_mt[16111]: WARNING: CameraProvider was not destroyed before client connection terminated.=== component_container_mt[16111]:          The client may have abnormally terminated. Destroying CameraProvider...=== component_container_mt[16111]: CameraProvider destroyed (0xffff7cb41320)=== component_container_mt[16111]: WARNING: Cleaning up 1 outstanding requests...=== component_container_mt[16111]: WARNING: Cleaning up 1 outstanding stream settings...=== component_container_mt[16111]: WARNING: Cleaning up 1 outstanding sessions...SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 466)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 533)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 466)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 533)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 466)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 533)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 466)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 533)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 466)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 533)
SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
=== component_container_mt[16111]: WARNING: Cleaning up 1 outstanding streams...(Argus) Error InvalidState: Unknown stream deleted. (in src/api/CaptureSessionImpl.cpp, function outputStreamDeleted(), line 1106)
=== component_container_mt[16111]: NOTE: Destroy all libargus objects before destroying the CameraProvider to avoid these warnings.=== component_container_mt[16111]: Connection cleaned up (FFFF846FB840)

Any support would be greatly appreciated!

Kind Regards
Sascha

[infiniteechorobotics]

@sascha8a I am having the same issue. Best I can tell the latest version of the isaac ros packages (3.2.4) are broken. You can try to build the 3.1 versions from source but I haven’t been able to get gxf to build properly. It seems that all of the packages need to be the same version so you can’t just install some with apt and build others. Still, if you do manage to build them please let me know. Otherwise we will have to wait for nvidia to provide the 3.1 binaries or fix the issue.

[infiniteechorobotics]

I got it working. Try this:

Argus Camera Segfault on Jetpack 6.2

I figured out how to get the binaries. Run these commands after entering the isaac ros docker: wget -qO - https://isaac.download.nvidia.com/isaac-ros/repos.key | sudo apt-key add - grep -qxF "deb https://isaac.download.nvidia.com/isaac-ros/release-3 $(lsb_release -cs) legacy-release-3.1" /etc/apt/sources.list || \ echo "deb https://isaac.download.nvidia.com/isaac-ros/release-3 $(lsb_release -cs) legacy-release-3.1" | sudo tee -a /etc/apt/sources.list sudo apt-get update sudo apt install ros-hu…

[Raffaello]

I forwarded your issue to the engineering. I keep you posted

[sascha8a]

Thank you for forwarding this issue to the engineering team. Are there any updates yet?

After further investigation, it seems, that this issue is not present in version 3.1.0, but present in 3.2.5.

The final last log-entries of the crashing node with version 3.2:

[component_container_mt-1] [INFO] [1741037567.636137066] [camera.argus_mono]: [NitrosNode] Node was started
[component_container_mt-1] 2025-03-03 21:32:47.854 WARN  extensions/hawk/components/argus_camera.cpp@948: Failed to get calibration data from Argus!

The final last log-entries of the non-crashing node with version 3.1:

[component_container_mt-1] [INFO] [1741038396.686877061] [camera.argus_mono]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1741038396.799523124] [camera.argus_mono]: [NitrosNode] Node was started
[ERROR] [component_container_mt-1]: process has died [pid 53014, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args -r __node:=container -r __ns:=/gate_detection'].

Please fix this issue, as it is a problem for us.

Thank you and best regards,
Sascha

[Raffaello]

Hi @sascha8a

I keep you posted when I’ve updates.

Best,
Raffaello