Jetson Orin Nano ISP limitations

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

Hi, following posts ISP max Resolution and Orin Nano ISP Processing Capability Evaluate - #8 by fuqiang1

The datasheet mentions support for raw Bayer sensors “up to 10 million pixels.” In another forum post, it is mentioned that the current support is up to “12000x9000.” Also, in the Orin Nano ISP processing capability thread, it was suggested that a sensor mode above 10 MP may be supported depending on CSI/VI limitations.

Because of these different references, the maximun resolution supported by the ISP and CSI/VI capture or bandwidth limits is not fully clear to me.

Could you clarify the following?

Is the “up to 10 million pixels” or the “12000x9000” frame resolution the limitation for the Orin Nano ISP?

If not, what are the actual ISP limits?

  • Maximum width and height

  • Maximum pixels per frame

  • Maximum pixels per second

  • Any RAW bit-depth limitations

Could you maybe provide some example configurations that showcase the ISP processing limits and CSI/VI capture limits on the Orin Nano?

Any help is appreciated.

Best regards.

3

*** Please note that this reply is generated by LLM automatically ***

The Jetson Orin Nano’s ISP (Image Signal Processor) capabilities and limitations are crucial for understanding its performance in various applications. Based on the provided context, I will attempt to clarify the ISP processing and capture limits for the Jetson Orin Nano.

ISP Limits:

  1. Maximum Input Frame Resolution: The statement “up to 10 million pixels” is not a hard maximum input frame resolution for the Orin Nano ISP. However, it is a suggested limit, and support for higher resolutions may be possible depending on the CSIVI (Camera Serial Interface and Video Input) limitations.
  2. Maximum Width and Height: The maximum width and height limits are not explicitly stated in the provided context. However, it is likely that the ISP can handle resolutions up to 4K (3840 x 2160) or higher, considering the CSIVI bandwidth limitations.
  3. Maximum Pixels per Frame: Assuming a 4K resolution (3840 x 2160), the maximum pixels per frame would be approximately 8.3 megapixels (3840 x 2160). However, this is not a hard limit, and the actual ISP limits may be higher or lower, depending on the CSIVI bandwidth and other factors.
  4. Maximum Pixels per Second: The maximum pixels per second (PPS) limit is not explicitly stated in the provided context. However, it is likely that the ISP can handle high frame rates (e.g., 30 FPS or higher) at lower resolutions, and lower frame rates at higher resolutions.
  5. RAW Bitdepth Limitations: The provided context does not mention any RAW bitdepth limitations for the Orin Nano ISP. However, it is likely that the ISP can handle RAW bitdepths up to 12 bits or higher, considering the typical RAW bitdepths used in modern image sensors.

Example Configurations:

To showcase the ISP processing and CSIVI capture limits on the Orin Nano, here are a few example configurations:

  1. High-Resolution Configuration: 4K resolution (3840 x 2160) at 30 FPS, with a RAW bitdepth of 12 bits. This configuration would push the ISP’s processing capabilities and CSIVI bandwidth limits.
  2. High-Frame-Rate Configuration: 1080p resolution (1920 x 1080) at 120 FPS, with a RAW bitdepth of 10 bits. This configuration would test the ISP’s ability to handle high frame rates and lower resolutions.
  3. Low-Light Configuration: 2MP resolution (1920 x 1080) at 30 FPS, with a RAW bitdepth of 14 bits. This configuration would evaluate the ISP’s performance in low-light conditions, where higher bitdepths are often required.

Comparison with Other Jetson Modules:

The Jetson Orin Nano’s ISP capabilities and limitations are likely to be similar to those of other Jetson modules, such as the Jetson Xavier NX and Jetson AGX Orin. However, the actual performance may vary depending on the specific module, CSIVI bandwidth, and other factors.

In summary, while the provided context does not offer explicit ISP limits for the Jetson Orin Nano, it suggests that the ISP can handle high resolutions, high frame rates, and high RAW bitdepths, depending on the CSIVI bandwidth limitations. The actual ISP limits and performance may vary depending on the specific use case and configuration.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

5

hello daniel.gonzalez3,

that comments (i.e. the current support is up to “12000x9000”) is the sensor we’ve enabled, instead of the ISP limitation.

let me reply several inquiries as below..

– for the image resolution,
there’s no limitation of the image height, but limitation of the image width.
the ISP maximum output width is 32766.

– for the pixel per frame and frames per second,
it depends-on the ISP throughput, you may check the ISP frequency, consider 2-pixel/clock and translates 15% off.
for example, Orin NX’s ISP clock is 1011.2Mhz, i.e. cat /sys/kernel/debug/clk/isp/clk_rate
it support up-to.. 1011.2 * 2 * 0.85 = 1719.04 ~= 1719 Mpixels per second.
so, assume it’s a 30-fps stream, it supports up-to 1719/30 = 57.3Mpixel per frame.

– RAW bit-depth limitations
please refer to Jetson Orin Series SoC Technical Reference Manual for [2.4.5 RAW Memory Formats] section. it uses T_R16 to handle Raw, it’s 16-bit unsigned word formats.
the maximum SDR bit-depth we’re supported is Raw 16-bits.
FYI, although PWL-WDR can support Raw 20-bits, ISP still having the maximum as 16-bits input, it’s PWL compressed/de-compress internally as 20-bit before processing.