Compressed video formats support

Hi all,

I have a couple of questions regarding to supported video formats on Jetson.

I am using an OV10640HDR camera on Jetson Xavier running JetPack 4.2.2 and FPDLinkIII SerDes configuration.

I have 2 camera modes, Long-Exposure Linear 12-bits and HDR 20b (compressed to 12b using PWL compression).

  • HDR and PWL support

The results of linear mode are better than HDR, I think some changes are needed to fully enable HDR mode but, is this PWL compression supported?

I am using hardware from D3 and they got issues with HDR mode, any information about this support will be appreciated.

  • Format for 2x12b mode

From linear mode, we got issues with bright images, so we are interested in a camera mode called 2x12b which merges long-exposure and short exposure channels to create an image with better results than single linear mode. At least this is the theory.

Linear mode uses the format MEDIA_BUS_FMT_SBGGR12_1X12, my understanding is 12 bits per pixel which matches the camera output.

Now, we are interested in the mode 2x12b, this mode uses 36 bits as input (3 channels with 12 bits each one: long exposure, short exposure and very short exposure) to 24 bits output from 2 channel. One saturated channel will be discarded (long exposure on bright images and very short on dark images).

Each used channel is mapped from 12 bits to 10 bits, these 2 bits are used for other purposes.

Which format should I used for 2x12b mode? it provides 24 bits per pixel but I’m not sure which format is correct. Since the data is compressed from 12b to 10b in each channel, is this compression supported?

For these cases, formats like MEDIA_BUS_FMT_SBGGR24_1X24 or MEDIA_BUS_FMT_SBGGR12_2X12 do not exist, so which is the correct procedure to receive data?

See below image of compression curve.

Regards.

Hi
This sensor tuning still on going with D3.

Hi Greivin,

We haven’t explored 2x12b mode. I am interested in NVIDIA’s comments on this mode.

As @ShaneCCC stated we are currently working with NVIDIA to resolve HDR issues on Xavier. An issue with PWL decompression on Xavier has been identified and a workaround provided. A few issues with our driver have been identified that we are also working on resolving. Although I can’t provide a timeline I can share that progress is being made and I think that we will have quality HDR output on Xavier. This is a very high priority at D3 right now.

While we work through this issue you might consider running this on Tx2. The PWL support on Tx2 is much better. If you enable HDR with our 2.0.0 BSP release you’ll see “reasonable” HDR output in bright scenes. In dark scenes you’ll see a green sparkling effect that looks like noise in dark regions. I expect this issue to be resolved as we work on the Xavier HDR issues.

As always feel free to contact us for further details.

Best Regards,
Greg

1 Like

Hi @ShaneCCC and @D3_growe,

Thanks for the update about the current status regarding the PWL compression.

Is there any information related to 2x12b mode?

From the kernel, I think the max bits-per-pixel supported format is 16 RAW, is this correct?

Information or ideas about how to handle 20bits linear data are appreciated.

Regards,
Greivin F.

The sensor does provide 12bit PWL WDR frame via CSI1 MIPI. Jetson platform receives the frame and decompresses to 16bpp to ISP.

Hi, My understanding is that there are two parts to get Tone Mapping/HDR working:

  1. PWL Decompression
  2. Tone mapping - Done by the built-in ISP.

Where is the current problem with Xavier? is it on the PWL decompression or is it on the Tone mapping? We will be happy to help in any of those or enhancing the driver since we have the documentation for this sensor.

Thanks,
-David

Hi @ShaneCCC and @D3_growe,

Just checking, is there any update about the HDR fix on Xavier for the OV10640 camera module?

Thanks.
Greivin F.

The tuning should be done.
@D3_growe Could you help to provide the latest driver and configure file.

1 Like

We do have some drive updates but it isn’t 100% complete yet. With the latest driver and a patch from NVIDIA the output is improved. It is still a work in progress but also a priority. I believe I am allowed to share the binary fix from NVIDIA. Let me verify that I can and I’ll provide the latest ov10640 driver code and the binary fix.

Hi,

Great!
Thanks for the update.

Greivin F.