syncStereo detecting sensors but module count 0 - no calibration data?

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
1

Hi,

Just a quick question on syncStereo sample. I have two IMX219 cameras attached via CSI and I get video out of both. I wanted to try stereo sync but the ouput of syncStereo is the following:

Executing Argus Sample: argus_syncstereo
Argus Version: 0.99.3.3 (multi-process)
getSensorPlacement for sensor 0 is SENSOR_PLACEMENT_FRONT_OR_TOP_O
getSensorPlacement for sensor 1 is SENSOR_PLACEMENT_REAR_OR_BOTTOM
Total Module Count is 0
PERF: performance thread contructed for #0 sensors
PRODUCER: Starting capture requests 
PRODUCER: Done -- exiting.

At first I suspected sensor placement should perhaps be LEFT and RIGHT instead of FRONT and READ but in a 2-camera system I read these are the only options.

So from the source I can glean that this means that iSyncSensorCalibrationData == NULL. Any ideas why this would be the case here? Is there no calibration data? Do I need to set calibration data first, and if so, how?

       printf("getSensorPlacement for sensor %d is %s\n",
                i, iCameraProperties->getSensorPlacement().getName());
        const Ext::ISyncSensorCalibrationData* iSyncSensorCalibrationData =
            interface_cast<const Ext::ISyncSensorCalibrationData>(cameraDevices[i]);
        if (iSyncSensorCalibrationData)
        {
            iSyncSensorCalibrationData->getSyncSensorModuleId(
                        syncSensorId, sizeof(syncSensorId));
            printf("Found : %s\n", syncSensorId);

This is Orin NX, Tegra 36.3, Updated Jetpack to 6.1, Seeed J401 carrier board.

3

I think the message could be ignore. Does the capture still successfully?

Thanks

4

Well, like in the output I posted it says Total Module Count = 0 and then exits, even though it finds two sensors, one front, one rear. So there’s no capturing going on. That is to say: the output I posted appears quick in succession (sample exits right away). I’m expecting I first need to see Found: <xx> as output (as indicated in the source) for the sample to work.

Or do you mean I should have a separate capture going on at this point, too (independent of running syncStereo). Perhaps I misunderstand the sample?

Thanks!

7

Could you check by below command.

./argus_syncsensor -l
8

Yes, output here:

2 Available CameraDevices:

  ==== CameraDevice 0: =========================================
    UUID:                      adca2c00,0f01,11e5,0001,00,00,00,00,00,00
    MaxAeRegions:              64
    MaxAwbRegions:             64
    FocusPositionRange:        [0, 0]
    LensApertureRange:
     2.400000
    IspDigitalGainRange:       [1.000000, 256.000000]
    ExposureCompensationRange: [-2.000000, 2.000000]
    NumSensorModes:            5
    SensorMode 0:
        Resolution:         3280x2464
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [47619048, 500000224]
                            (2.00 to 21.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 1:
        Resolution:         3280x1848
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [35714284, 500000224]
                            (2.00 to 28.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 2:
        Resolution:         1920x1080
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [33333334, 500000224]
                            (2.00 to 30.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 3:
        Resolution:         1640x1232
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [33333334, 500000224]
                            (2.00 to 30.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 4:
        Resolution:         1280x720
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [16666667, 500000224]
                            (2.00 to 60.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No

  ==== CameraDevice 1: =========================================
    UUID:                      adca2c00,0f01,11e5,0000,00,00,00,00,00,00
    MaxAeRegions:              64
    MaxAwbRegions:             64
    FocusPositionRange:        [0, 0]
    LensApertureRange:
     2.400000
    IspDigitalGainRange:       [1.000000, 256.000000]
    ExposureCompensationRange: [-2.000000, 2.000000]
    NumSensorModes:            5
    SensorMode 0:
        Resolution:         3280x2464
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [47619048, 500000224]
                            (2.00 to 21.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 1:
        Resolution:         3280x1848
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [35714284, 500000224]
                            (2.00 to 28.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 2:
        Resolution:         1920x1080
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [33333334, 500000224]
                            (2.00 to 30.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 3:
        Resolution:         1640x1232
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [33333334, 500000224]
                            (2.00 to 30.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 4:
        Resolution:         1280x720
        HdrRatioRange:  [1.000000, 1.000000]
        ExposureTimeRange:  [13000, 683709000]
        FrameDurationRange: [16666667, 500000224]
                            (2.00 to 60.00 fps)
        AnalogGainRange:    [1.000000, 10.625000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No

However, the following might be a clue (in case the histogram needs to be executed). When I run ./argus_syncsensor without -l it gives the following error:

Argus Version: 0.99.3.3 (multi-process)
PRODUCER: Creating left stream.
PRODUCER: Creating right stream.
PRODUCER: Launching disparity checking consumer
Initializing CUDA
CONSUMER: Connecting CUDA consumer to left stream
CONSUMER: Connecting CUDA consumer to right stream
CONSUMER: Waiting for Argus producer to connect to left stream.
PRODUCER: Starting repeat capture requests.
CONSUMER: Waiting for Argus producer to connect to right stream.
CONSUMER: Streams connected, processing frames.
Failed to launch histogram_smem_atomics kernel (error code the provided PTX was compiled with an unsupported toolchain.)!

(argus_syncsensor:1778758): Gtk-WARNING **: 11:19:09.002: cannot open display: 
WARNING Argus: 5 client objects still exist during shutdown:
	281472710414656 (0xaaaaf92ea3b0)
	281472712558912 (0xaaaaf92e83a0)
	281472714838320 (0xaaaaf92e8550)
	281472716813904 (0xaaaaf92e9600)
	281472716844464 (0xaaaaf92e8300)

This may be part of my situation (explained in Jetpack 6.1 - docker cuda version mismatch - #5 by AastaLLL) that I can only execute Upgrade Compute Stack on a J401 carrier board (without dist-upgrade which means I have Jetpack 6.1 on a 36.3 Tegra now and no updated driver. nvidia-smi reports it supports up to CUDA 12.2 while everything else in the system is upgraded to Jetpack 6.1 and CUDA 12.6.

9

The syncStereo need calibration data for it.

11

This sample code need specific HW design for sync signal suppose imx219 don’t support it.

Thanks

12

Ok, that’s fair enough, understood. The IMX2019 is a very simple device that I just used as a starting point to check whether I could get CSI to work correctly for 2 cameras on the carrier board before deciding on a setup with better cameras to actually do stereo vision on.

If you say other (better) cameras will have specific HW design and thereby come with calibration data, and this is what will make the sample work, that’s perfect. Thanks!

16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.