Issues with YUV Camera Development Using the SIPL Framework on JetPack 7.2

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

Hello everyone.

I am developing the ISX031+96717 camera with the SIPL framework on Jetpack 7.2. When I set the format to YUV422 in the JSON file, the format fails to be recognized by the program, whereas RAW10, RAW12 and RAW16 work fine.

I would like to know what source‑code modifications are required to enable support for the YUV422 format. Below are my JSON file settings along with the program execution output.

“cameraConfigs”: [

    {

        "name": "ISX031_ONE_CAM",

        "type": "GMSL",

        "description": "Single ISX031 on CSI-AB link 0 (stock binary + -H)",

        "cameraModules": \[

            {

                "name": "MODULE_ISX031",

                "linkIndex": 0,

                "transportIndex": 0

            }

        \],

        "transportSettings": \[

            {

                "name": "transportSettings_jetson-thor_AB"

            }

        \]

    },

    {

        "name": "MODULE_ISX031",

        "type": "GMSL",

        "description": "module definition (ISX031 1920x1536 @ 30fps, 6G)",

        "moduleDriverName": "R0SIS031S5RU1197NB3",

        "serInfo": {

            "name": "MAX96717",

            "i2cAddress": "0x40"

        },

        "linkMode": "LINK_MODE_GMSL2_6GBPS",

        "fsyncMode": "osc_manual",

        "mipiSettings": {

            "dphyRate": 2500000,

            "phyMode": "dphy"

        },

        "sensorInfo": \[

            {

                "name": "ISX031",

                "description": "ISX031 YUV sensor stub (UYVY 1920x1536 @ 30fps)",

                "i2cAddress": "0x1A",

                "numContext": 1,

                "virtualChannels": \[

                    {

                        "cfa": "uyvy",

                        "embeddedTopLines": 0,

                        "embeddedBottomLines": 0,

                        "inputFormat": "yuv422",

                        "width": 1920,

                        "height": 1536,

                        "fps": 30.0,

                        "isEmbeddedDataTypeEnabled": false

                    }

                \]

            }

        \]

    }

\]

t@t:/usr/src/jetson_sipl_api/sipl/json$ sudo nvsipl_query -t module_isx031.json -c ISX031_ONE_CAM
[sudo] password for t:
nvsipl_query version 1.0.0
Initializing Camera Query API…
Database parsed successfully.
Parsed JSON file: module_isx031.json
=== SensorSystemConfig: ISX031_ONE_CAM ===
Transports (2):
Transport: transportSettings_jetson-thor_AB (ID: 0)
Description: GMSL CSI-AB with external MAX96724
GMSL Transport | CSI Port: GH | I2C Bus: 10 | I2C Port: 0 | Passive: No
TX Port: 1 | Power Port: 0
Deserializer: Max96724GmslDeserializer (I2C: 0x27)
Power Deser: TegraDeserPowerDriver-NativeGpio (GPIO: 9043968)
Transport: transportSettings_jetson-thor_CD (ID: 1)
Description: GMSL CSI-CD with external MAX96724
GMSL Transport | CSI Port: CD | I2C Bus: 9 | I2C Port: 0 | Passive: No
TX Port: 1 | Power Port: 0
Deserializer: Max96724GmslDeserializer (I2C: 0x27)
Power Deser: TegraDeserPowerDriver-NativeGpio (GPIO: 3211264)

Modules (1):
Module: R0SIS031S5RU1197NB3
Description: module definition (ISX031 1920x1536 @ 30fps, 6G)
Platform: | Config: | Transport ID: 0
MIPI: DPHY | Lanes: 4 | DPHY Rate: 2500000 kbps | Clock: 2500000
Type: GMSL | Link: 0 | Mode: GMSL2_6G | Fsync: OSC_Manual
Serializer: MAX96717 (I2C: 0x40)
Sensor: ISX031 (ID: 0)
I2C Address: 0x1a
Trigger Mode: No | Contexts: 1 | Group: 0
VC[0]: src=4294967295 dst=4294967295
Resolution: 1920x1536 @ 30 fps
Input Format: RAW12 | CFA: Unknown Type
Embedded Data: No (Top: 0, Bottom: 0)

Done

3

Hi,

I have seen some SIPL configuration examples where the input format is specified as:

"inputFormat": "422"

So I recommend trying the following configuration first:

{
    "cfa": "uyvy",
    "inputFormat": "422",
    "embeddedTopLines": 0,
    "embeddedBottomLines": 0,
    "width": 1920,
    "height": 1536,
    "fps": 30.0,
    "isEmbeddedDataTypeEnabled": false
}

Then run nvsipl_query again and check whether the reported input format changes from RAW12.

If the input format is reported correctly as YUV422 but the CFA is still shown as Unknown Type, you could also try:

"cfa": "yuyv",
"inputFormat": "422"

This would be useful as a diagnostic test to determine whether the JSON parser recognizes YUV pixel-order strings such as uyvy or yuyv.

Regards

Manuel Leiva
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com

4

I’ve attempted both of these configuration entries, yet neither "cfa": "yuyv" nor "inputFormat": "422" gets recognized by the program.

5

This doc might be helpful. sgx_yuv_gmsl2.md

To look at this locally:

git clone --depth 1 --filter=blob:none --sparse \
    https://github.com/SENSING-Technology/nvidia-jetson-camera-drivers.git

cd nvidia-jetson-camera-drivers

git sparse-checkout set \
    'Jetson Orin Nano&NX Devkit/SG4A-NONX-G2Y-A1/JetPack7.2/SG4A_NONX_G2Y_A1_Orin_NANO_SIPL_JP7.2_L4TR39.2'

In this file nvidia-jetson-camera-drivers/jetson-orin-nano-devkit/SG4A-NONX-G2Y-A1/JetPack7.2/SG4A_NONX_G2Y_A1_Orin_NANO_SIPL_JP7.2_L4TR39.2/query/sg4a_nonx_g2y_a1/sgx_yuv_gmsl2.json

This json is extracted from that file and some of following might help:

{
        "deserializers": [
                {
                        "name": "MAX96712_Fusa_nv_CD",
                        "description": "Maxim MAX96712 GMSL2 Deserializer - CSI-CD",
                        "i2cAddress": "0x6b"
                }
        ],
        "serializers": [
                {
                        "name": "MAX96717F",
                        "description": "Maxim 96717F Serializer",
                        "i2cAddress": "0x40",
                        "useCDIv2API": true
                },
                {
                        "name": "MAX96717",
                        "description": "Maxim 96717 Serializer",
                        "i2cAddress": "0x40",
                        "useCDIv2API": true
                }
        ],
        "imageSensors": [
                {
                        "name": "GW5",
                        "description": "GW5 YUV Sensor (UYVY output)",
                        "i2cAddress": "0x6D",
                        "virtualChannels": [
                                {
                                        "cfa": "uyvy",
                                        "embeddedTopLines": 0,
                                        "embeddedBottomLines": 0,
                                        "inputFormat": "yuv422",
                                        "width": 3840,
                                        "height": 2160,
                                        "fps": 30.0,
                                        "isEmbeddedDataTypeEnabled": false
                                },
                                {
                                        "cfa": "uyvy",
                                        "embeddedTopLines": 0,
                                        "embeddedBottomLines": 0,
                                        "inputFormat": "yuv422",
                                        "width": 1920,
                                        "height": 1536,
                                        "fps": 30.0,
                                        "isEmbeddedDataTypeEnabled": false
                                }
                        ],
                        "isTriggerModeEnabled": true,
                        "isTPGEnabled": false,
                        "useCDIv2API": true
                }
        ],
        "platformTransportSettings": [
                {
                        "name": "jetson-orin-nano",
                        "description": "Jetson Orin Nano Devkit (p3834)",
                        "boardIdPrefix": "p3834",
                        "enableMasks": ["0x1111"],
                        "transportSettings": [
                                {
                                        "name": "transportSettings_jetson-orin-nano_CD",
                                        "type": "GMSL",
                                        "description": "GMSL CSI-CD with external MAX96712",
                                        "csiPort": "csi-cd",
                                        "phyMode": "dphy",
                                        "desTxPort": 2,
                                        "deserInfo": {
                                                "name": "Max96712GmslDeserializer",
                                                "i2cAddress": "0x6b"
                                        },
                                        "powerControlInfo": {
                        "deserializerInfo": {
                            "name": "TegraDeserPowerDriver",
                            "gpioPinIndex": 9043968
                        }
                                        },
                                        "i2cDevice": 9,
                                        "desI2CPort": 0,
                                        "groupInitProg": false
                                }
                        ]
                }
        ],
        "cameraConfigs": [
                {
                        "name": "SGX_YUV_GMSL2_3M",
                        "type": "GMSL",
                        "description": "SGX-YUV-GMSL single module definition (GW5 YUV sensor 1920x1536)",
                        "moduleDriverName": "SGX_YUV_GMSL2",
                        "serInfo": {
                                "name": "MAX96717",
                                "i2cAddress": "0x40"
                        },
                        "linkMode": "LINK_MODE_GMSL2_6GBPS",
                        "fsyncMode": "osc_manual",
                        "mipiSettings": {
                                "dphyRate": 2500000,
                                "phyMode": "dphy"
                        },
                        "sensorInfo": [
                                {
                                        "name": "GW5",
                                        "description": "GW5 YUV Sensor (UYVY 1920x1536 @ 30fps)",
                                        "i2cAddress": "0x1A",
                                        "numContext": 1,
                                        "virtualChannels": [
                                                {
                                                        "cfa": "uyvy",
                                                        "embeddedTopLines": 0,
                                                        "embeddedBottomLines": 0,
                                                        "inputFormat": "yuv422",
                                                        "width": 1920,
                                                        "height": 1536,
                                                        "fps": 30.0,
                                                        "isEmbeddedDataTypeEnabled": false
                                                }
                                        ]
                                }
                        ]
                },
        ]
}
6

I have tried this approach. When I use the libnvcamerahal.so and libnvuddf_sgx_yuv_gmsl2_cameramodule_library.so provided from the git repository together with this JSON configuration, the program can correctly recognize the YUV422 format. However, once I replace libnvuddf_sgx_yuv_gmsl2_cameramodule_library.so with my own built‑out sensor driver library, YUV422 recognition stops working. That is why I am seeking help. I could not locate where the image format is configured within the SIPL source code.

10

Do you confirm the DriverInfo?

https://uddf-docs.nvidia.com/sipl_integration.html#driver-configuration-matching

11

Could you let me know what information you need me to confirm? I can’t open this link properly.

12

https://docs.nvidia.com/jetson/archives/r38.4/DeveloperGuide/SD/CameraDevelopment/CoECameraDevelopment/SIPL-for-L4T/SIPL-UDDF-Integration.html

13

This information has been confirmed. DriverInfo.name can match the content in the JSON file.