Get image metadata in isaac_ros_argus_camera

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hello. I am using a Jetson Orin Nano with a MIPI CSI camera and the isaac_ros_argus_camera package.

My application requires the actual white balance coefficients (Red, Green-even, Green-odd, Blue multipliers) and exposure applied by the ISP for every frame to ensure accuracy in downstream processing.

I understand that libargus provides per-frame metadata via ICaptureMetadata. However, since the core driver logic is encapsulated in the compiled binary libgxf_isaac_argus.so, I cannot modify it to publish this data.

I have three specific questions:

  1. Metadata Retrieval: Does NVIDIA provide a way to surface the applied white balance gains and exposure from the ArgusCamera GEM as a ROS topic? Note that I need the actual multipliers used for the transform, not just the scene’s color temperature or illuminant estimate (as returned by getAwbWbEstimate).

  2. Manual Coefficient Control: Can the standard isaac_ros_argus_camera node be configured to accept individual R, Gr, Gb, B channel gains as ROS parameters (calling the underlying setWbGains API)? Currently, only wbmode presets appear to be exposed. While not ideal, I would be willing to implement the exposure and WB controllers myself, as that would bring this data into my program easily.

  3. Managed NITROS Reference: If the current GEM is a “black box” regarding this metadata, is there a Managed NITROS reference example for Orin that shows how to implement a custom libargus producer? I need to know how to pack the GPU buffer and the metadata struct into a single NITROS-compatible message to maintain a zero-copy pipeline.

Thank you for your assistance.

2

Hello @marc100,

Thanks for posting in the Isaac ROS forum!

Argus::ICaptureMetadata exposes the information you want on a per‑frame basis:

  • AWB gains used for this capture as a BayerTuple<float>: [R, Gr, Gb, B] via getAwbGains().

  • Sensor exposure time via getSensorExposureTime() (nanoseconds).

  • Sensor analog gain via getSensorAnalogGain() and ISP digital gain via getDigitalGain().

So at the LibArgus layer you can retrieve “actual multipliers used” per frame.
However, what ArgusCamera GEM used by isaac_ros_argus_camera currently only publishes: Image data (Nitros image → ROS image topics) and CameraInfo and some transforms. It does not publish the data from the above APIs on any ROS topic, and there is no parameter to turn that on.
Since the core Argus GXF codelet is in libgxf_isaac_argus.so, you cannot inject extra publishers or modify how it uses ICaptureMetadata from the outside.
You could implement your own LibArgus‑based capture that calls ICaptureMetadata::getAwbGains(), getSensorExposureTime(), etc., and then publish a custom ROS message alongside the image.

There is no public NITROS+Argus+metadata example yet for your reference. But you could try to integrate your own LibArgus to grab frames and metadata, publishing your per‑frame metadata on a parallel ROS topic, and add the NitrosImage forward node from isaac_ros_nitros_image_type into your launch to convert ROS image into a NitrosImage for the rest of the NITROS pipeline.
Note: this is not yet a validated implementation, but a suggested approach based on your current requirement only.