Dear Forum,
I am attempting to port isaac_ros_argus_camera v0.9.3-ea3 (together with isaac_ros_common with same version) for the custom CSI camera pair with Jetson Orin NX and Jetpack-5.1.2. I know that later versions of isaac_ros_argus_camera should be better aligned with the recent Jetpack releases (that support the Orin family), but I want to keep/use the implementation aligned with our own implementation based on early Jetpacks, and especially to avoid dependency on the isaac_ros_nitros and the gxf recently introduced.
So far, I was able to run the argus_monocular node :
~/Workspace/Nvidia/ros_ws$ ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="rgb8"
[INFO] [1731042038.293170637] [argus_monocular]: Argus Version: 0.99.3.3 (multi-process)
[INFO] [1731042038.293803341] [argus_monocular]: Set the camera model using the node parameter "device"
[INFO] [1731042038.293856845] [argus_monocular]: Set the sensor mode using the node parameter "sensor"
[INFO] [1731042038.293904525] [argus_monocular]: Set the output image format using the node parameter "output_encoding". Supported encodings: "mono8" and "rgb8"
[INFO] [1731042038.293944525] [argus_monocular]: Following camera model indices are available:
[INFO] [1731042038.294156749] [argus_monocular]:
[INFO] [1731042038.294181869] [argus_monocular]: 0 imx296_centerright
[INFO] [1731042038.294225549] [argus_monocular]: Sensor modes supported for this camera device:
[INFO] [1731042038.294553069] [argus_monocular]: 0 (1456 x 1088)
[INFO] [1731042038.294780301] [argus_monocular]: 1 (1280 x 720)
[INFO] [1731042038.295006829] [argus_monocular]:
[INFO] [1731042038.295053805] [argus_monocular]: 1 imx296_bottomright
[INFO] [1731042038.295083693] [argus_monocular]: Sensor modes supported for this camera device:
[INFO] [1731042038.297454957] [argus_monocular]: 0 (1456 x 1088)
[INFO] [1731042038.297730061] [argus_monocular]: 1 (1280 x 720)
[INFO] [1731042038.297854253] [argus_monocular]:
[INFO] [1731042038.298126381] [argus_monocular]: Capturing from device 0 using sensor mode 0 (1456x1088)
[WARN] [1731042038.298344237] [argus_monocular]: Cannot get ISyncSensorCalibrationData interface
[INFO] [1731042038.407708813] [argus_monocular]: Starting repeat capture requests.
[INFO] [1731042039.135632109] [argus_monocular]: Consumer Running
and could see the following node :
~/Workspace/Nvidia/ros_ws$ ros2 topic list
/camera_info
/image_raw
/image_raw/compressed
/image_raw/compressedDepth
/image_raw/theora
/parameter_events
/rosout
and
:~/Workspace/Nvidia/ros_ws $ ros2 topic hz /image_raw
WARNING: topic [/image_raw] does not appear to be published yet
average rate: 29.130
min: 0.025s max: 0.049s std dev: 0.00609s window: 31
average rate: 29.031
min: 0.025s max: 0.049s std dev: 0.00487s window: 60
average rate: 28.618
min: 0.025s max: 0.049s std dev: 0.00466s window: 88
average rate: 28.626
min: 0.023s max: 0.049s std dev: 0.00448s window: 117
average rate: 28.551
min: 0.023s max: 0.049s std dev: 0.00452s window: 146
average rate: 28.593
min: 0.023s max: 0.049s std dev: 0.00424s window: 175
average rate: 28.564
min: 0.020s max: 0.054s std dev: 0.00433s window: 204
average rate: 28.525
min: 0.020s max: 0.054s std dev: 0.00414s window: 233
average rate: 28.516
min: 0.020s max: 0.054s std dev: 0.00414s window: 262
average rate: 28.502
min: 0.020s max: 0.054s std dev: 0.00403s window: 291
average rate: 28.490
min: 0.020s max: 0.054s std dev: 0.00386s window: 320
average rate: 28.476
min: 0.019s max: 0.057s std dev: 0.00418s window: 349
However, I was unable to visualize the /image_raw topic using the Image View of the RQT Visualization plugin.
Could someone help, please ?
Attached below are the patches of modification.
isaac_ros_argus_camera-ea3.txt (8.3 KB)
isaac_ros_common-ea3.txt (1.3 MB)
Hi @khang.l4es
Thank you for your post. Unfortunately, I cannot verify your setup on an older Jetpack release.
I can give some hints, that may can help to detect where the error is coming from.
- Have you tried with other visualizers? Like foxglove or ROS Package: image_view
- if you plot the output with
ros 2 topic echo /image_raw --raw do you see an output or an empty matrix?
Best,
Raffaello
Hi @Raffaello,
Please find my answers inline :
Have you tried with other visualizers? Like foxglove or ROS Package: image_view
~/Workspace/Nvidia/ros_ws$ ros2 run image_view image_view image:=/image_raw
[WARN] [1731079862.675096930] [rcl]: Found remap rule 'image:=/image_raw'. This syntax is deprecated. Use '--ros-args --remap image:=/image_raw' instead.
[INFO] [1731079863.042485254] [image_view_node]: Using transport "raw"
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.2.0) ../modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
[ros2run]: Aborted
~/Workspace/Nvidia/ros_ws$ ros2 run image_view image_view --ros-args --remap image:=/image_raw
[INFO] [1731079900.563867123] [image_view_node]: Using transport "raw"
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.2.0) ../modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
[ros2run]: Aborted
if you plot the output with ros 2 topic echo /image_raw --raw do you see an output or an empty matrix?
~/Workspace/Nvidia/ros_ws$ ros2 topic echo /image_raw --raw
b'\x00\x01\x00\x00\x97/.g\x8dvL\x1a\x06\x00\x00\x0010592\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xa3:F\x1d\x06\x00\x00\x0010593\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xf8\xa7? \x06\x00\x00\x0010594\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xf9y/#\x06\x00\x00\x0010595\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\x83\x85#&\x06\x00\x00\x0010596\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xa3\x11\x13)\x06\x00\x00\x0010597\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xa3<\x11,\x06\x00\x00\x0010598\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xd6\xd4\t/\x06\x00\x00\x0010599\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\x8cz\xf41\x06\x00\x00\x0010600\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xc5\xc0\xef4\x06\x00\x00\x0010601\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xb9W\xe87\x06\x00\x00\x0010602\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x97/.g\xd8d\xd7:\x06\x00\x00\x0010603\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.g\x14I9\x02\x06\x00\x00\x0010604\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.g\x87\x931\x05\x06\x00\x00\x0010605\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.g\xd9\xc3)\x08\x06\x00\x00\x0010606\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.g]\x10\x1b\x0b\x06\x00\x00\x0010607\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.g\xd1\xb8\x13\x0e\x06\x00\x00\x0010608\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.gX\x92\x15\x11\x06\x00\x00\x0010609\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
---
b'\x00\x01\x00\x00\x98/.gLf\xff\x13\x06\x00\x00\x0010610\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00rgb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
Hi again @Raffaello,
While waiting for your help of the isaac_ros_argus_camera v0.9.3-ea3, I switched to test the isaac_ros_argus_camera v2.0.0 (being aligned to Jetpack-5.1.2 that I am using) which in turns requires isaac_ros_common and isaac_ros_nitros of the same version but requires no modification to be compiled successfully.
However, when launching the isaac_ros_argus_camera_mono, I faced another issue :
~/Workspace/Nvidia$ ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_mono.launch.py
[INFO] [launch]: All log files can be found below /home/cti/.ros/log/2024-11-09-16-29-20-966299-VINS125-95357
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [95370]
[component_container_mt-1] [INFO] [1731144561.508900256] [argus_mono_container]: Load Library: /home/cti/Workspace/Nvidia/ros_ws/install/isaac_ros_argus_camera/lib/libmono_node.so
[component_container_mt-1] [INFO] [1731144561.560694597] [argus_mono_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusMonoNode>
[component_container_mt-1] [INFO] [1731144561.560840491] [argus_mono_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusMonoNode>
[component_container_mt-1] [INFO] [1731144561.567041913] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1731144561.567310595] [argus_mono]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1731144561.568446031] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1731144561.576149271] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1731144561.580636003] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1731144561.585559968] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1731144561.591081684] [NitrosContext]: [NitrosContext] Loading application: '/home/cti/Workspace/Nvidia/ros_ws/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1731144561.592433480] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1731144561.597071034] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] 2024-11-09 16:29:21.597 WARN gxf/std/program.cpp@514: No system specified. Nothing to do
**[component_container_mt-1] [INFO] [1731144561.602530540] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_argus.so**
**[component_container_mt-1] [ERROR] [1731144561.624891847] [NitrosContext]: [NitrosContext] dlopen failed when opening "/home/cti/Workspace/Nvidia/ros_ws/install/isaac_ros_gxf/share/isaac_ros_gxf/gxf/lib/libgxf_argus.so": desktop-shell.so: cannot open shared object file: No such file or directory**
[component_container_mt-1] [ERROR] [1731144561.624996107] [ArgusNitrosContext]: loadExtensions Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1731144561.625962128] [argus_mono]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1731144561.626025234] [argus_mono]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-11-09 16:29:21.626 ERROR gxf/std/program.cpp@533: Attempted interrupting when not running (state=0).
[component_container_mt-1] 2024-11-09 16:29:21.626 ERROR gxf/core/runtime.cpp@1400: Graph interrupt failed with error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [ERROR] [1731144561.626085653] [argus_mono]: [NitrosContext] GxfGraphInterrupt Error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [INFO] [1731144561.626097621] [argus_mono]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1731144561.626118454] [argus_mono]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1731144561.626141847] [argus_mono]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1731144561.627199615] [argus_mono]: [NitrosNode] Application termination done
[component_container_mt-1] [ERROR] [1731144561.631032339] [argus_mono_container]: Component constructor threw an exception: loadExtensions Error: GXF_FAILURE
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'argus_mono' of type 'nvidia::isaac_ros::argus::ArgusMonoNode' in container '/argus_mono_container': Component constructor threw an exception: loadExtensions Error: GXF_FAILURE
Hi @khang.l4es
I think it is something related to this error:
I don’t know if you setup the gxf, but maybe can be rebuilt: GitHub - NVIDIA-ISAAC-ROS/gxf at release-2.0.0 following the version 2.0.0
1 Like