Is it possible to get P010_10LE format from nvaraguscamerasrc?

Hi everyone,

I’ve been looking at JetPack 5.0.2 that the hardware encoders report the P010_10LE format, here is what gst-inspect-1.0 reports:

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 format: { (string)I420, (string)NV12, (string)P010_10LE, (string)NV24 }
              framerate: [ 0/1, 2147483647/1 ]

However, when looking for nvarguscamerasrc, the gst-inspect-1.0 reports:

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 format: { (string)NV12 }
              framerate: [ 0/1, 2147483647/1 ]

Question1 for NVIDIA: Is there any way to obtain the P010_10LE format from nvaraguscamerasrc to feed the encoder in that format?
Question2 for NVIDIA: Can the P010_10LE buffers be obtained from the ISP?
Question3 for NVIDIA: Can be possible to get P010_10LE buffers from the ISP without using GStreamer? Is this a hardware or software limitation?

Thanks,

Eduardo.

Sorry to tell nvarguscamerasrc only support Bayer format.

Hi Shane. We can see that nvarguscamerasrc currently only outputs NV12. Could you please comment on the other questions asked, perhaps more is possible using the underlying Jetson Multimedia API?

Q1 → No possible
Q2 → No
Q3 → No, ISP only output NV12/YUV420/Bayer

According to the AGX Orin data sheet, it contains an “ISP (up to 1.85 GPix/s) with HDR support”.
How can we configure the ISP to support HDR if it only outputs in 8-bit pixel formats NV12 and YUV420?

Hi @ShaneCCC,

Thank you for the provided information, could you please elaborate a little more about this NVIDIA statement related to the Xavier NX?

NVIDIA Xavier NX Response → ISP output pixel format is YUV 8-bit by default. it’s supported to configure as PIXEL_FMT_P016 as 16-bit YUV formats.
URL: Argus image pixel format

Is P010 not supported but P016 supported? If this is correct, nvarguscamerasrc can be modified to provide P016. Please correct me if I am wrong.

Regards.

Here is the Argus support. Have a check the RELEASE.TXT in MMAPI.

  • Current EGLStream Buffer Format Support:

    Format State
    ======================= ==========
    PIXEL_FMT_Y8 Not Supported
    PIXEL_FMT_Y16 Not Supported
    PIXEL_FMT_YCbCr_420_888 Works with all consumers.
    PIXEL_FMT_YCbCr_422_888 Not Supported
    PIXEL_FMT_YCbCr_444_888 Not Supported
    PIXEL_FMT_JPEG_BLOB Not Supported (TODO: Remove format)
    PIXEL_FMT_RAW16 Works with all consumers

@ShaneCCC It seems the README is out of date since PIXEL_FMT_P016 is also supported? There is an example in the MMAPI Argus samples - argus_gstvideoencode which gives the option of P016, but when that flag is enabled, the following error occurs:

NvMMLiteVideoEncDoWork:NvEncNvVicConvertSurfaceFormat failed
NvVideoEncTransferOutputBufferToBlock: DoWork failed line# 680 
NvVideoEnc: NvVideoEncTransferOutputBufferToBlock TransferBufferToBlock failed Line=691

This makes sense since the example is trying to pass P016 straight into an h26x encoder, when they explicitly only support one of I420, NV12, P010_10LE, NV24.

Please can you provide a patch to fix this broken example, and provide specific instructions/ examples for how to use the P016 Argus output with the VIC/ encode/ decode engines, which only work with 8 or 10 bit formats?

1 Like

Hi,
On Jetpack 5, the supported format is YUV420 8-bit. Other formats are not supported.

@DaneLLL other formats are not supported by what - Argus? MMAPI? VIC/ video encoders?

If so does this mean HDR imaging is fundamentally not possible with Argus?

Hi,

Argus supports YUV420 8-bit only on Jetpack 5. The format YUV420 10-bit is not supported.

@DaneLLL I can get images from the rawBayerOutput Argus sample using PIXEL_FMT_RAW16 which apparently is running ISP. However, there is this warning in the nvargus-daemon logs: (NvCameraIspMergeExt) Error BadParameter: NvColorSpace_NonColor is set for isp for isp capture. Does mean that ISP does not work with 16 bit frames? If so, then this is yet another example application which has been shipped as broken. Otherwise, presumably we can debayer and convert to P010 for use in h26x encoders or whatever else?

But right now we cannot get a clear answer on whether it is possible for Argus to output > 8 bit output from its ISP. There is conflicting information on this forum and from MMAPI samples.

Hi,
There is no complete software stacks for > 8 bit output format. For a complete solution, it needs support in camera functionality and ISP tuning. It requires significant effort in supporting the cases, so we focus on the case of 8 bit output format.

@DaneLLL In that case, can you be specific about what is and isn’t supported in terms of a “complete solution”:

  • is ISP / camera tuning supported for 16-bit bayered frames (PIXEL_FMT_RAW16)?
  • is ISP / camera tuning supported for debayered 16-bit P016 (PIXEL_FMT_P016)?

If not then why are there examples for it in the MMAPI?

Following that, it possible to convert any16-bit formats to 10-bit using the VIC or iGPU, which would then allow these frames to be accepted by hardware encoders?

Looks like you are misunderstand the PIXEL_FMT_RAW16, it is ISP output raw data doesn’t matter with tunning. Current we only support 10/12 bit Bayer sensor.

Suppose you can use CUDA for debayer PIXEL_FMT_RAW16 and convert to 10bit.

@ShaneCCC can you elaborate please - does this mean that there’s no tuning in terms of say colour correction, but there should still be ISP in terms of auto-exposure etc?

As mentioned above, when I run the rawBayerOutput example, which uses PIXEL_FMT_RAW16 with and without ISP, this error message occurs in the Argus daemon logs:

(NvCameraIspMergeExt) Error BadParameter: NvColorSpace_NonColor is set for isp for isp capture

When I debayer the image which was captured with ISP, it is very underexposed. Both of these things lead me to believe that there is no ISP running on PIXEL_FMT_RAW16 output even when ISP is enabled?

Sorry for not clear your question here.

Did you check the sensor gain/exposure for PIXEL_FMT_RAW16 capture.

@ShaneCCC @DaneLLL Let me change tack slightly, and go directly back to the original question from @EduardoSalazar96 .

I saw recently that were was explicit support for P010 caps in nvarguscamerasrc in JetPack4.6.2, via PIXEL_FMT_P016 and the NvBufferColorFormat_NV12_10LE color format. However, this has been removed in JetPack 5.0.2. We have tried porting these flags to the JP5.0.2 nvarguscamerasrc but we get an error in the createNvBuffer function, which returns a status of STATUS_INVALID_SETTINGS.

Can you explain why support for P010 was removed from nvarguscamerasrc between JetPack 4.6 and 5, and whether it will be re-introduced in a later JetPack 5 release?

Hi,
For now there is no plan to support P010_10LE on Jetpack 5. The explanation is in the comment and we would like to focus on maintaining camera functionality and ISP tuning for 8-bit use-cases.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.