ArgusMonoNode GPU NV12→RGB8 conversion takes ~22ms at 4000×3000 on Orin Nano, VIC engine crashes

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hardware/Software:

  • Jetson Orin Nano Super Developer Kit (nvidia,p3768-0000+p3767-0005-super)
  • JetPack 6.2.1 (L4T R36.4.7)
  • Isaac ROS isaac_ros_argus_camera 3.2.5 (ros-humble-isaac-ros-argus-camera)
  • GXF Argus extension 3.2.11 (ros-humble-gxf-isaac-argus)
  • VIC: nvidia,tegra234-vic
  • Kernel: 5.15.148-tegra
  • Sensor: Sony IMX586 (CSI, imx586 9-001a) in mode 1 (4000×3000 @ 72fps)
  • Power mode: MAXN_SUPER, clocks pinned via clocks.sh

Issue Summary:

I’m using ArgusMonoNode to capture from an IMX586 at 4000×3000 (mode 1, 72fps). This is equivalent to an IMX477-class 4K×3K mode. I have a minimal test setup — just the ArgusMonoNode publishing and a trivial NITROS subscriber that counts frames and logs FPS (no processing, no inference, no display). Despite this, I’m only getting ~44 fps with the GPU converter, and the VIC converter crashes immediately.

Problem 1: VIC engine_type fails at 4000×3000

Setting engine_type: “VIC” in argus_camera_mono_node.yaml causes an immediate crash on the first frame:

ERROR ./gxf/extensions/tensorops/components/ConvertColorFormat.cpp@83: color conversion operation failed.
ERROR ./gxf/extensions/tensorops/components/TensorOperator.cpp@233: operation failed.
ERROR gxf/std/entity_executor.cpp@596: Failed to tick codelet color_converter_operator in entity: converter_left code: GXF_FAILURE

The Tegra234 SoC has a VIC engine, and the Isaac ROS documentation lists engine_type: “VIC” as a valid option specifically intended for offloading color conversion from the GPU. All Jetson Orin modules should have this hardware. Is VIC not supported at this resolution, or is there a configuration step I’m missing?

Problem 2: GPU NV12→RGB8 conversion is the bottleneck at ~22ms median

With engine_type: “GPU” (the default), the GXF converter_left entity takes a median of 21.4ms per tick to convert NV12→RGB8. This alone exceeds the 13.9ms frame budget for 72fps and is the sole bottleneck in the pipeline. For comparison, when I bypass the converter entirely by subscribing in NV12 format, I get a rock-solid 72 fps with the camera capture tick at only 13.6ms median.

Benchmark Results (all with clocks.sh, MAXN_SUPER):Benchmark Results (all with jetson_clocks.sh, MAXN_SUPER):

Test Output Format Steady-State FPS Converter Tick (median) Camera Tick (median)
GPU + RGB8 nitros_image_rgb8 ~44 fps 21.4ms 18.6ms
NV12 passthrough nitros_image_nv12 72 fps 0.05ms (bypassed) 13.6ms
VIC + RGB8 nitros_image_rgb8 N/A (crash) crash on first frame —

Without clocks.sh, the GPU test starts at ~72fps but degrades to ~38fps over ~60 seconds as thermal throttling kicks in, with the converter tick climbing from 23ms to 27ms.

GXF JobStatistics (GPU + RGB8, max clocks, 2663 frames):

Entity State Count Median 90th % Max
argus_camera Ticking 2663 18.59ms 20.75ms 188.48ms
converter_left (GPU) Ticking 2661 21.39ms 24.80ms 75.00ms
splitter_left Ticking 2662 0.15ms 0.18ms 4.68ms

GXF JobStatistics (NV12 passthrough, max clocks, 4161 frames):

Entity State Count Median 90th % Max
argus_camera Ticking 4161 13.63ms 13.88ms 189.13ms
converter_left (bypass) Ticking 4160 0.05ms 0.05ms 0.30ms
splitter_left Ticking 4160 0.17ms 0.20ms 0.58ms

Test Setup:

The subscriber is a minimal ManagedNitrosSubscriber that increments a counter and logs FPS every 30 frames — it performs zero processing on the image data. The full pipeline is just ArgusMonoNode → NITROS topic → FrameLoggerNode, running as composable nodes in a single component_container_mt.

Questions:

  1. Is engine_type: “VIC” expected to work at 4000×3000 on Orin Nano, or is there a resolution/format limitation? The documentation doesn’t mention any such restriction.
  2. Is ~22ms for a GPU NV12→RGB8 conversion at 4000×3000 (36MB output frame) expected on the Orin Nano’s GPU? This seems high for what should be a straightforward colorspace conversion — that’s only ~1.6 GB/s throughput on a GPU with ~68 GB/s memory bandwidth.
  3. Is there a recommended way to reduce the converter overhead? We need RGB8 downstream for our vision pipeline. Would a custom CUDA kernel or cv::cuda::cvtColor be faster than the GXF TensorOps path?
8

Hello @marc100,

Thanks for posting in the Isaac ROS forum!
I’ll address your questions below.

  1. Besides setting engine_type to "VIC", did you also change output_encoding or any other related parameters? With the VIC backend, not all format pairs are supported. You can refer to the VPI Convert Image Format documentation and its backend-support matrix. In particular, when VIC is selected, only certain conversions are implemented (for example, NV12→RGBA8 is supported, while NV12→RGB8 is not, which is what the default ArgusMonoNode + NITROS setup). If the requested input/output encoding pair is not in the supported set for VIC, VPI returns VPI_ERROR_INVALID_IMAGE_FORMAT and, in Isaac ROS, this surfaces as:
    color conversion operation failed in ConvertColorFormat.cpp.
    That error almost always means VPI rejected the conversion (unsupported or incompatible format/params), not that the kernel itself crashed.
  2. From the data you presented, it doesn’t seem to be a hardware limit but an artifact of the current Isaac ROS / GXF / TensorOps pipeline overhead.
    Have you tried following these steps to set the GPU and CPU clock to max and set the to power to MAX settings? If not, please give it a try.
    Also, please refer to this thread, Isaac ROS Argus Camera is meant to work with specific cameras whose vendors have provided camera drivers that are compatible with NVIDIA’s libArgus. Note: there could be a performance penalty compared against an Argus-compatible camera + driver.
  3. Yes, you could try to implement a custom CUDA node to do the conversion directly on the GPU buffer instead with persistent allocations and minimal synchronization.
9

Thank you for the pointer — you were right that the default NV12→RGB_U8 pair isn’t supported by VIC. We got it working with NV12→RGBA_U8 after two changes, but VIC turns out to be significantly slower than GPU at this resolution.

Changes to enable VIC

argus_camera_mono_node.yaml:

  • output_type: “RGB_U8” → “RGBA_U8”
  • engine_type: “GPU” → “VIC”
  • Added allocate_pitch_linear: true to the ImageAdapter — without this, the output buffer pitch (16000 bytes) doesn’t meet VIC’s 256-byte alignment requirement (needs 16128). This parameter defaults to false.

isaac_ros_argus_camera_mono_spec.yaml:

  • Added nitros_image_nv12 → nitros_image_rgba8 to StreamConvertColorFormat supported_formats
  • Added nitros_image_rgba8 to MessageSplitter supported_formats

Results

All tests on Orin Nano Super, MAXN_SUPER, jetson_clocks, 4000×3000 @ 72fps.

GPU with RGB8 output: converter median 21.4ms, ~44 fps steady state. VIC with RGBA8 at default 115 MHz: converter median 37.6ms, ~26 fps. VIC with RGBA8 at max 435 MHz: converter median 37.4ms, ~27 fps. NV12 passthrough (no conversion): converter bypassed at 0.05ms, full 72 fps.

Questions

  1. Is allocate_pitch_linear expected to be required with VIC? It seems like this should either auto-detect based on engine type or be documented as a prerequisite.
  2. VIC conversion time is ~37ms regardless of clock speed (115 MHz vs 435 MHz). Is VIC expected to be this slow at 12MP, or does the clock insensitivity suggest something else is bottlenecking the conversion?
10

After receiving the feedback about VIC format support, we investigated further and found that the GXF pipeline was adding massive overhead to the conversion itself. We wrote a minimal NITROS subscriber that receives NV12 from ArgusMonoNode and converts it standalone using two backends: NPP (GPU) and VPI VIC (called directly, not through GXF).

To get VPI VIC working standalone, we needed: VPI_IMAGE_FORMAT_NV12_ER as the input format (what Argus actually outputs), VPI_IMAGE_FORMAT_RGBA8 as output, and 256-byte aligned output pitch. Images were created via vpiImageCreateWrapper with VPI_BACKEND_VIC and updated per-frame via vpiImageSetWrapper.

Standalone conversion results (all at 4000×3000, MAXN_SUPER, max clocks including VIC at 435 MHz):

NPP on GPU (nppiNV12ToRGB_8u_P2C3R_Ctx): 2.4ms average, 72fps.

VPI VIC standalone (vpiSubmitConvertImageFormat with VPI_BACKEND_VIC): 14.8ms average, ~60fps.

Both produce correct frames (verified via PNG output). For comparison, the same conversions through the GXF pipeline were 21.4ms (GPU) and 37.4ms (VIC). So GXF is adding ~19ms overhead on top of GPU conversion and ~23ms on top of VIC conversion. The tradeoff for our application: NPP gives full 72fps but uses GPU cycles. VPI VIC gives ~60fps but leaves the GPU completely free for inference. We’ll likely go with VIC since freeing the GPU is more valuable than the extra 12fps.

One remaining question: Is ~15ms expected for VIC NV12_ER→RGBA8 at 4000×3000 (12MP) on Orin Nano? VIC clock speed had no effect in our earlier GXF tests (37ms at both 115 MHz and 435 MHz), but standalone it’s consistently ~15ms at max clocks. We haven’t tested standalone at lower VIC clocks to see if it scales.

11

Thanks for sharing the detailed test results.
We don’t currently have internal performance numbers at the 12 MP resolution you’re using, but for reference our VPI benchmark for NV12_ER→RGBA8 at 1080p on the VIC backend is around 0.82 ms (±0.0007 ms). If it’s based on simple pixel scaling from that data, ~15 ms at 12 MP is indeed higher than we’d expect if only raw VIC throughput were the limit. Currently, VPI performance on Orin Nano is already under active investigation.
And regarding your suggestion to make allocate_pitch_linear auto-detect based on engine type (or clearly document it as a prerequisite), we agree this would make things clearer. We’ve passed this feedback to the internal team for review for a future release and documentation update.
Thanks again for the thorough analysis and for helping us improve the stack.

12

One final suggestion which may clear up some of the performance benchmarking for an orin nano is that none of the benchmarks in VPI are avaliable for the orin nano. I understand you are currently working on it. From my uninformed perspective, I have very little idea of how to compare the performance across different hardware. My best guess was to look at the max clock speed and say that, worst case, the orin nano can be (orin nano hz)/(agx orin hz) time slower than benchmark. Given that, these are way larger errors than expected. However, that could be a completely incorrect estimation, so a rough way to approximate performance from benchmarks would be quite helpful! Obviously there is so much complexity, determining where most of the time is taken up, and scaling factors based on the biggest blockers, so not looking for a specific or detailed measurement.

13

Thanks for the thoughtful feedback. You’re right that performance in Isaac ROS could be quite complex considering that it could include conversion paths, engines, memory/layout, power modes, etc., so simple clock-ratio scaling can only ever be a rough approximation. Within Isaac ROS we do provide dedicated benchmarks for a number of packages (via isaac_ros_benchmark), and for the NITROS Bridge specifically you can reference the official benchmark setup and results in the Performance section. We appreciate your suggestion about clearer guidance on how to compare performance across hardware, and we’ll use this feedback to improve our documentation going forward.