Disable ISP(Not use .nito) in sample_camera demo

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
[√] DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[√] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
[√] DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
[√] other

Host Machine Version
[√] native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi,
How can i disable the ISP(not use the .nito)when running the sample_camera in DriveWorks?

Dear @zhaohm8,
Is the requirement is to get raw data and save? Could you check using output-format=raw in rig.json in sample_camera and see if works for your case?

Hi @SivaRamaKrishnaNV ,
Is the requirement is to get raw data and save?
→ yes
Could you check using output-format=raw in rig.json in sample_camera and see if works for your case?
→ i use output-format=raw in rig.json with camera ‘F008A120RM0A’, but it doesn’t work;
And i use output-format=raw in rig.json with YUV422 camera, but it also doesn’t work

Could you share the rig.json and used sample_camera command used to save RAW data. Also, please share the complete logs in case you see any issue.

Hi @SivaRamaKrishnaNV ,
The rig.json is following:

{
"rig": {
    "sensors": [
        {
            "name": "camera:sample0",
            "nominalSensor2Rig_FLU": {
                "roll-pitch-yaw": [
                    0.0,
                    0.0,
                    0.0
                ],
                "t": [
                    1.8621,
                    -0.1939,
                    1.3165
                ]
            },
            "parameter": "camera-name=F008A120RM0A,interface=csi-e,CPHY-mode=1,link=1,output-format=raw,async-record=1,file-buffer-size=16777216",
            "properties": {
                "Model": "ftheta",
                "bw-poly": "0.000000000000000 5.35356812179089e-4 4.99266072928606e-10 4.27370422037554e-12 -6.68245573791717e-16",
                "cx": "1927.764404",
                "cy": "1096.686646",
                "height": "2168",
                "width": "3848"
            },
            "protocol": "camera.gmsl"
        }
    ],
    "vehicle": {
        "valid": false
    }
},
"version": 2

}

The log is following:

nvidia@tegra-ubuntu:/usr/local/driveworks/samples/bin$ ./sample_camera --rig=/usr/local/driveworks/data/samples/sensors/camera/camera/rig_raw.json --write-file=/works/recorder/F008A120RM0A_1.raw

ProgramArguments: Missing argument ‘dwTracePath’ requested
[24-10-2022 19:19:04] Platform: Detected Drive Orin P3710
[24-10-2022 19:19:04] TimeSource: monotonic epoch time offset is 1666577470086841
[24-10-2022 19:19:04] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[24-10-2022 19:19:04] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[24-10-2022 19:19:04] PTP Time is available from Eth Driver
[24-10-2022 19:19:04] Adding variable DW_Base:DW_Version
[24-10-2022 19:19:04] Addded variable DW_Base:DW_Version
[24-10-2022 19:19:04] Platform: number of GPU devices detected 1
[24-10-2022 19:19:04] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_04_12_05, Arch: ga10b
[24-10-2022 19:19:04] Platform: currently selected GPU device integrated ID 0
[24-10-2022 19:19:04] CUDLAEngine:getDLACountandCheck: CUDLA version is = 1002001
[24-10-2022 19:19:04] CUDLAEngine:getDLACountandCheck: Number of DLA devices = 2
[24-10-2022 19:19:04] ========== DLA Device 0 Attributes ==========
[24-10-2022 19:19:04] UVA supported = 0
[24-10-2022 19:19:04] Device version = 1
[24-10-2022 19:19:04] Device 0 Check PASSED
[24-10-2022 19:19:04] ========== DLA Device 1 Attributes ==========
[24-10-2022 19:19:04] UVA supported = 0
[24-10-2022 19:19:04] Device version = 1
[24-10-2022 19:19:04] Device 1 Check PASSED
[24-10-2022 19:19:04] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘./resources’: VirtualFileSystem: Failed to mount ‘./resources/resources.pak’
[24-10-2022 19:19:04] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.8/samples/bin/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.8/samples/bin/data/resources.pak’
[24-10-2022 19:19:04] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks/data
[24-10-2022 19:19:04] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks/data/resources.pak’
[24-10-2022 19:19:04] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.8/data
[24-10-2022 19:19:04] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.8/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.8/data/resources.pak’
[24-10-2022 19:19:04] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks/lib/libdw_base.so.5.8
[24-10-2022 19:19:04] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.8/targets/aarch64-Linux/lib/libdw_base.so.5.8
[24-10-2022 19:19:04] SDK: No resources(.pak) mounted, some modules will not function properly
[24-10-2022 19:19:04] SDK: Create NvMediaDevice
[24-10-2022 19:19:04] SDK: use EGL display as provided
[24-10-2022 19:19:04] TimeSource: monotonic epoch time offset is 1666577470086841
[24-10-2022 19:19:04] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[24-10-2022 19:19:04] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[24-10-2022 19:19:04] PTP Time is available from Eth Driver
[24-10-2022 19:19:04] Initialize DriveWorks SDK v5.8.82
[24-10-2022 19:19:04] Release build with GNU 9.3.0 from buildbrain-branch-0-g33d6f4a1440 against Drive PDK v6.0.5.0
[24-10-2022 19:19:04] SensorFactory::createSensor() → time.nvpps, nvpps-device=/dev/nvpps0
[24-10-2022 19:19:04] Can not create time sensor: DW_HAL_CANNOT_OPEN_CHANNEL: EndpointNVPPS: cannot open NVPPS device with name: /dev/nvpps0
Error: Permission denied
[24-10-2022 19:19:04] Rig::fromFile: Loading rig file: /usr/local/driveworks/data/samples/sensors/camera/camera/rig_raw.json
[24-10-2022 19:19:04] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
[24-10-2022 19:19:04] No valid data file found for camera:sample0 in parameter string: camera-name=F008A120RM0A,interface=csi-e,CPHY-mode=1,link=1,output-format=raw,async-record=1,file-buffer-size=16777216 (using configuration folder /usr/local/driveworks/data/samples/sensors/camera/camera/)
[24-10-2022 19:19:04] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
onInitialize: creating camera.gmsl with params: camera-name=F008A120RM0A,interface=csi-e,CPHY-mode=1,link=1,output-format=raw,async-record=1,file-buffer-size=16777216
[24-10-2022 19:19:04] SensorFactory::createSensor() → camera.gmsl, camera-name=F008A120RM0A,interface=csi-e,CPHY-mode=1,link=1,output-format=raw,async-record=1,file-buffer-size=16777216
[24-10-2022 19:19:04] CameraGMSLSelector: missing ‘output-format=processed/yuv’ in the sensor parameters, frames might be captured with bad sensor settings(AE/AWB) and reprocess may result image quality issues
[24-10-2022 19:19:04] CameraBase: pool size set to 8
[24-10-2022 19:19:04] SensorFactory::createSensor() → camera.gmsl.master,
[24-10-2022 19:19:04] CameraMaster::parseDevBlock Getting device info list.
[24-10-2022 19:19:04] devBlock: 0 Slave = 0 Interface = csi-e Camera_name = F008A120RM0A Link = 1
[24-10-2022 19:19:04] Camera Match Name: F008A120RM0A Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295 linkIndex: 4294967295 serInfo.Name: MAX9295
[24-10-2022 19:19:04] Client, Setting up information for camera ID 0
[24-10-2022 19:19:04] Client, successfully found info for camera ID 0 bound to id 0
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] ImageStreamer(NvMedia → CUDA)
[24-10-2022 19:19:04] CameraClient: using NITO found at /opt/nvidia/nvmedia/nit/F008A120RM0A.nito
Processed not selected as master output format parameter. No images will be previewed.
[24-10-2022 19:19:04] SIPLMaster::SIPLMaster: Setting up master camera
[24-10-2022 19:19:04] Platform:
[24-10-2022 19:19:04] Platform Config:
[24-10-2022 19:19:04] Description:
[24-10-2022 19:19:04] Number of device blocks: 1
[24-10-2022 19:19:04] Device Block : 0
[24-10-2022 19:19:04] csiPort: 6
[24-10-2022 19:19:04] i2cDevice: 2
[24-10-2022 19:19:04] Deserializer Name: MAX96712
[24-10-2022 19:19:04] Deserializer Description: Maxim 96712 Aggregator
[24-10-2022 19:19:04] Deserializer i2cAddress: 41
[24-10-2022 19:19:04] Simulator Mode: 0
[24-10-2022 19:19:04] Passive Mode: 0
[24-10-2022 19:19:04] Phy Mode: 1
[24-10-2022 19:19:04] Number of camera modules: 1
[24-10-2022 19:19:04] CameraModule index: 1
[24-10-2022 19:19:04] Name :F008A120RM0A
[24-10-2022 19:19:04] Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295
[24-10-2022 19:19:04] Serializer name: MAX9295
[24-10-2022 19:19:04] Serializer description: Maxim 9295 Serializer
[24-10-2022 19:19:04] Serializer i2cAdress: 98
[24-10-2022 19:19:04] EEPROM name: N24C64
[24-10-2022 19:19:04] EEPROM i2cAddress: 84
[24-10-2022 19:19:04] EEPROM description: N24C64 EEPROM
[24-10-2022 19:19:04] Sensor ID: 0
[24-10-2022 19:19:04] Sensor name: AR0820
[24-10-2022 19:19:04] Sensor description: OnSemi AR0820 Sensor
[24-10-2022 19:19:04] Sensor i2cAddress: 16
[24-10-2022 19:19:04] Sensor isTPGEnabled: 0
[24-10-2022 19:19:04] Sensor isTriggerMode: 1
[24-10-2022 19:19:04] cfa: 35
[24-10-2022 19:19:04] embeddedTopLines: 6
[24-10-2022 19:19:04] embeddedBottomLines: 0
[24-10-2022 19:19:04] inputFormat: 8
[24-10-2022 19:19:04] height: 2168
[24-10-2022 19:19:04] width: 3848
[24-10-2022 19:19:04] fps: 30.000000
[24-10-2022 19:19:04] Embedded Data: 1
[24-10-2022 19:19:04] CameraMaster available outputs: 2
[24-10-2022 19:19:04] CameraClient: setting pipeline config for camera ID 0
[24-10-2022 19:19:04] CameraMaster: master initiation
MAX96712: Revision 5 detected
MAX96712 Link 1: PHY optimization was enabled
MAX9295: Revision 8 detected!
Sensor AR0820 GRBG Rev 2.1 detected!
[24-10-2022 19:19:05] CameraMaster: Bootstrap NvSIPL and ClientPipelines complete
[24-10-2022 19:19:05] CameraClient::readEEPROMInfo:
[24-10-2022 19:19:05] mmIntrinsicModelID: 27
[24-10-2022 19:19:05] mmIntrinsicCoeff[0]: -0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[1]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[2]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[3]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[4]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[5]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[6]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[7]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[8]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[9]: 1.892143e+11
[24-10-2022 19:19:05] mmIntrinsicCoeff[10]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[11]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[12]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[13]: 0.000000
[24-10-2022 19:19:05] mmIntrinsicCoeff[14]: 0.000170
[24-10-2022 19:19:05] mmIntrinsicCoeff[15]: 8.000000
[24-10-2022 19:19:05] mmIntrinsicMeasuredFlagID: 237
[24-10-2022 19:19:05] fwVersionMajor: 0
[24-10-2022 19:19:05] fwVersionMinor: 0
[24-10-2022 19:19:05] fwVersionPatch: 0
[24-10-2022 19:19:05] SAL: bootstrap complete
[24-10-2022 19:19:05] Initialize DriveWorks VisualizationSDK v5.8.82
[24-10-2022 19:19:05] Initialize DriveWorksGL SDK v5.8.82
onInitialize: Total cameras 1
[24-10-2022 19:19:05] CameraClient: camera params: format=raw,type=disk,file=/works/recorder/F008A120RM0A_1.raw
[24-10-2022 19:19:05] EncoderRAW: Reading header information
[24-10-2022 19:19:05] EncoderRAW: encoding SIPL info
[24-10-2022 19:19:05] Streamer NVM/CPU using NvSciBuffer
Main: Starting master.
[24-10-2022 19:19:05] CameraMaster: starting…
[24-10-2022 19:19:05] CameraClient: allocating image pool for ICP
[24-10-2022 19:19:05] Allocating NvSci group
[24-10-2022 19:19:05] CameraClient: Failed to register plugin!6
[24-10-2022 19:19:05] Driveworks exception thrown: DW_NVMEDIA_ERROR: CameraClient: Failed to register plugin!

terminate called after throwing an instance of ‘std::runtime_error’
what(): [2022-10-24 19:19:05] DW Error DW_NVMEDIA_ERROR executing DW function:
dwSensor_start(m_camera[i])
at /usr/local/driveworks/samples/src/sensors/camera/camera/main.cpp:449
Aborted

Are you using the “Entron F008A120RM0AES” module mentioned in https://developer.nvidia.com/drive/ecosystem-orin#camera? If yes, have you referred to those rig files for it under /usr/local/driveworks/ before attempting to modify them for the raw output format?

Hi @VickNV ,
Are you using the “Entron F008A120RM0AES” module mentioned in https://developer.nvidia.com/drive/ecosystem-orin#camera?
→ yes
If yes, have you referred to those rig files for it under /usr/local/driveworks/ before attempting to modify them for the raw output format?
→ yes, I can use the camera and get some images with output-format=process

To assist you further, could you please provide the sample_camera command, the rig file, and the output from running the application with the “process” output-format?

Hi @VickNV ,
To assist you further, could you please provide the sample_camera command, the rig file, and the output from running the application with the “process” output-format?
->The log and rig file are as following.

log.txt (10.5 KB)
rig.txt (1.1 KB)

Thank you for sharing the log and rig files. It appears that the settings are working correctly.

However, for further clarification, could you please try using the parameter settings in the following snippet from /usr/local/driveworks/data/samples/sensors/camera/camera/rig.json and connect your camera module accordingly? Afterward, if it works, please attempt to modify it to write to a raw file based on these settings.

"parameter": "camera-name=F008A120RM0AV2,interface=csi-ab,CPHY-mode=1,link=0,output-format=processed,async-record=1,file-buffer-size=16777216",

Hi @VickNV
However, for further clarification, could you please try using the parameter settings in the following snippet from /usr/local/driveworks/data/samples/sensors/camera/camera/rig.json and connect your camera module accordingly?
->It is working correctly, the log and rig file are as following.
rig.txt (1.1 KB)
log.txt (10.4 KB)

Afterward, if it works, please attempt to modify it to write to a raw file based on these settings.
->It is working wrong.
rig.txt (1.1 KB)
log_raw.txt (10.8 KB)

Modify the rig.file “output-format=raw” and write to a raw file.
→ It is working wrong.
rig_raw.txt (1.1 KB)
log_raw.txt (10.7 KB)

I’m check this with our team and will get back to you. Thanks.

Based on your logs, why are you still using DRIVE OS 6.0.5? Please upgrade to 6.0.8.1 and have a try. Thanks.

Hi @VickNV
Based on your logs, why are you still using DRIVE OS 6.0.5?
→ Because the customer is using this version, I can’t upgrade it right now

I’ll check this with our team and get back to you.

This specific use case has been tested and validated on the upcoming release. I recommend waiting for the release to address this issue effectively.