Driveworks sample_camera throws error

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi,
I am using the driveworks sample_camera application for quanta camera. I have updated the rig.json accordingly.
I was successfully able to run nvsipl_camera and record in raw,yuv formats.
But when I run driveworks, i face this problem
nvidia@tegra-ubuntu:/usr/local/driveworks/samples/bin$ ./sample_camera --offscreen=2
sample_camera !
dw_samples::SamplesDataPath::get() : /usr/local/driveworks-5.10/data
ProgramArguments: Missing argument ‘dwTracePath’ requested
WindowEGL: find EGL devices
WindowEGL: found 1 EGL devices
WindowEGL: init EGL with GLES3 context
WindowEGL: bind OpenGLES3 API
WindowEGL: offscreen mode → use EGL PBuffer surface 1280x800
WindowEGL: create EGL context
WindowEGL: assign EGL context to current thread
[04-10-2023 12:26:35] Platform: Detected Drive Orin P3710
[04-10-2023 12:26:35] TimeSource: monotonic epoch time offset is 1696407401634844
[04-10-2023 12:26:35] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[04-10-2023 12:26:35] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[04-10-2023 12:26:35] PTP Time is available from Eth Driver
[04-10-2023 12:26:35] Adding variable DW_Base:DW_Version
[04-10-2023 12:26:35] Added variable DW_Base:DW_Version
[04-10-2023 12:26:35] Platform: number of GPU devices detected 1
[04-10-2023 12:26:35] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_05_10_03, Arch: ga10b
[04-10-2023 12:26:35] Platform: currently selected GPU device integrated ID 0
[04-10-2023 12:26:35] CUDLAEngine:getDLACount: CUDLA version is = 1003000
[04-10-2023 12:26:35] CUDLAEngine:getDLACount: Number of DLA devices = 2
[04-10-2023 12:26:35] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘./resources’: VirtualFileSystem: Failed to mount ‘./resources/resources.pak’
[04-10-2023 12:26:35] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.10/samples/bin/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.10/samples/bin/data/resources.pak’
[04-10-2023 12:26:35] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks/data
[04-10-2023 12:26:35] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks/data/resources.pak’
[04-10-2023 12:26:35] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.10/data
[04-10-2023 12:26:35] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.10/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.10/data/resources.pak’
[04-10-2023 12:26:35] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks/lib/libdw_base.so.5.10
[04-10-2023 12:26:35] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.10/targets/aarch64-Linux/lib/libdw_base.so.5.10
[04-10-2023 12:26:35] SDK: No resources(.pak) mounted, some modules will not function properly
[04-10-2023 12:26:35] SDK: use EGL display as provided
[04-10-2023 12:26:35] TimeSource: monotonic epoch time offset is 1696407401634844
[04-10-2023 12:26:35] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[04-10-2023 12:26:35] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[04-10-2023 12:26:35] PTP Time is available from Eth Driver
[04-10-2023 12:26:35] Initialize DriveWorks SDK v5.10.87
[04-10-2023 12:26:35] Release build with GNU 9.3.0 from buildbrain-branch-0-g9a5b4670e12 against Drive PDK v6.0.6.0
[04-10-2023 12:26:35] SensorFactory::createSensor() → time.nvpps, nvpps-device=/dev/nvpps0
[04-10-2023 12:26:35] Can not create time sensor: DW_HAL_CANNOT_OPEN_CHANNEL: EndpointNVPPS: cannot open NVPPS device with name: /dev/nvpps0
Error: Permission denied
[04-10-2023 12:26:35] fromFile: Loading rig file: /usr/local/driveworks-5.10/data/samples/sensors/camera/camera/rig.json
[04-10-2023 12:26:35] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
[04-10-2023 12:26:35] No valid data file found for camera:sample3 in parameter string: camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-ab,CPHY-mode=1,link=0,output-format=processed,async-record=1,file-buffer-size=16777216 (using configuration folder /usr/local/driveworks-5.10/data/samples/sensors/camera/camera/)
[04-10-2023 12:26:35] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
onInitialize: creating camera.gmsl with params: camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-ab,CPHY-mode=1,link=0,output-format=processed,async-record=1,file-buffer-size=16777216
[04-10-2023 12:26:35] SensorFactory::createSensor() → camera.gmsl, camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-ab,CPHY-mode=1,link=0,output-format=processed,async-record=1,file-buffer-size=16777216
[04-10-2023 12:26:35] CameraBase: pool size set to 8
[04-10-2023 12:26:35] SensorFactory::createSensor() → camera.gmsl.master,
[04-10-2023 12:26:35] CameraMaster::parseDevBlock Getting device info list.
[04-10-2023 12:26:35] devBlock: 0 Slave = 0 Interface = csi-ab Camera_name = HZKJ_IMX728_ES2_V2_120FOV Link = 0 GroupInit = 0 RecCfg = 1
[04-10-2023 12:26:35] Camera Match Name: HZKJ_IMX728_ES2_V2_120FOV Description: QUANTA IMX728 ES2 V2 RGGB module - 120-deg FOV, MIPI-IMX728, MAX96717 linkIndex: 4294967295 serInfo.Name: MAX96717F
[04-10-2023 12:26:35] Client, Setting up information for camera ID 0
[04-10-2023 12:26:35] Client, successfully found info for camera ID 0 bound to id 0
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] ImageStreamer(NvMedia → CUDA)
[04-10-2023 12:26:35] CameraClient: using NITO found at /usr/share/camera/HZKJ_IMX728_ES2_V2_120FOV.nito
[04-10-2023 12:26:35] SIPLMaster::SIPLMaster: Setting up master camera
[04-10-2023 12:26:35] Platform:
[04-10-2023 12:26:35] Platform Config:
[04-10-2023 12:26:35] Description:
[04-10-2023 12:26:35] Number of device blocks: 1
[04-10-2023 12:26:35] Device Block : 0
[04-10-2023 12:26:35] csiPort: 2
[04-10-2023 12:26:35] i2cDevice: 0
[04-10-2023 12:26:35] Deserializer Name: MAX96712
[04-10-2023 12:26:35] Deserializer Description: Maxim 96712 Aggregator
[04-10-2023 12:26:35] Deserializer i2cAddress: 41
[04-10-2023 12:26:35] Simulator Mode: 0
[04-10-2023 12:26:35] Passive Mode: 0
[04-10-2023 12:26:35] Phy Mode: 1
[04-10-2023 12:26:35] Number of camera modules: 1
[04-10-2023 12:26:35] CameraModule index: 0
[04-10-2023 12:26:35] Name :HZKJ_IMX728_ES2_V2_120FOV
[04-10-2023 12:26:35] Description: QUANTA IMX728 ES2 V2 RGGB module - 120-deg FOV, MIPI-IMX728, MAX96717
[04-10-2023 12:26:35] Serializer name: MAX96717F
[04-10-2023 12:26:35] Serializer description: Maxim 96717F Serializer
[04-10-2023 12:26:35] Serializer i2cAdress: 64
[04-10-2023 12:26:35] EEPROM name: N24C64
[04-10-2023 12:26:35] EEPROM i2cAddress: 84
[04-10-2023 12:26:35] EEPROM description: N24C64 64-Kbit serial I2C EEPROM
[04-10-2023 12:26:35] Sensor ID: 0
[04-10-2023 12:26:35] Sensor name: IMX728
[04-10-2023 12:26:35] Sensor description: Sony IMX728 Sensor
[04-10-2023 12:26:35] Sensor i2cAddress: 28
[04-10-2023 12:26:35] Sensor isTPGEnabled: 0
[04-10-2023 12:26:35] Sensor isTriggerMode: 1
[04-10-2023 12:26:35] cfa: 33
[04-10-2023 12:26:35] embeddedTopLines: 1
[04-10-2023 12:26:35] embeddedBottomLines: 20
[04-10-2023 12:26:35] inputFormat: 8
[04-10-2023 12:26:35] height: 2160
[04-10-2023 12:26:35] width: 3840
[04-10-2023 12:26:35] fps: 30.000000
[04-10-2023 12:26:35] Embedded Data: 0
[04-10-2023 12:26:35] CameraMaster available outputs: 1
[04-10-2023 12:26:35] CameraClient: setting pipeline config for camera ID 0
[04-10-2023 12:26:35] CameraMaster: master initiation
MAX96712: Revision 5 detected
MAX96712 Link 0: PHY optimization was enabled
[04-10-2023 12:26:36] CameraMaster: master init failed with status = 10
[04-10-2023 12:26:36] Driveworks exception thrown: DW_UNEXPECTED_EVENT: CameraMaster::bootstrap failed with sipl status 10

terminate called after throwing an instance of ‘std::runtime_error’
what(): [2023-10-04 12:26:36] DW Error DW_UNEXPECTED_EVENT executing DW function:
dwSAL_start(m_sal)
at /usr/local/driveworks/samples/src/sensors/camera/camera/main.cpp:363
Aborted (core dumped)

2

Dear @arkos,
Is it possible to check the issue on our recent release DRIVE OS 6.0.8.1?

4

Dear @arkos,
Also, could you confirm if the camera is listed in DRIVE AGX Orin Sensors & Accessories | NVIDIA Developer ? If so, please share camera details and used rig.json file as well.
From the log, I can see you are using Quanta sensor. Did you check with Quanta on this issue?

5

Hi Siva,

Thanks for the reply.
I am kind of unsure if the rig.json that is created is correct or not.
I have connected to GroupA quad port 3. Hence in rig.json I have mentioned it to be like
“parameter”: “camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-a,CPHY-mode=1,link=3,output-format=processed”.
Also I want to extend this with 6 cameras.
{
“rig”: {
“sensors”: [
{
“name”: “camera:sample0”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
0.0,
0.0,
0.0
],
“t”: [
1.8621,
-0.1939,
1.3165
]
},
“parameter”: “camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-a,CPHY-mode=1,link=3,output-format=processed”,
“properties”: {
“Model”: “ftheta”,
“bw-poly”: “0.000000000000000 5.35356812179089e-4 4.99266072928606e-10 4.27370422037554e-12 -6.68245573791717e-16”,
“cx”: “1927.764404”,
“cy”: “1096.686646”,
“height”: “2168”,
“width”: “3848”
},
“protocol”: “camera.gmsl”
}
],
“vehicle”: {
“valid”: false
}
},
“version”: 2
}

6

Yes we ensured to choose the supported camera.

image
image1343×73 7.85 KB

nvmedia/nvsipl_camera works like a charm. But not Driveworks.

7

Dear @arkos,
Could you share the Nvsipl_camera command and log?

9

This is not the standard binary but with my changes, nevertheless even the original binary also works correctly.

sudo ./nvsipl_camera_csv_epochtime -c HZKJ_IMX728_ES2_V2_120FOV_CPHY_x4 --link-enable-masks “0x1000 0x0000 0x0000 0x0000” --showfps --enableRaw --disableISP1Output --disableISP2Output
MAX96712: Revision 5 detected
MAX96712 Link 3: PHY optimization was enabled
MAX96717: Revision 4 detected!
Driver Version : HZKJ_IMX728_ES2-V3NK6061-BASE602-v1.5
Sensor IMX728 Rev 49 RGGB detected!
IMX728SetDeviceConfig[420]Quanta write Calibration Data
IMX728SetDeviceConfig[431]Quanta write MNR Data
Toggle 0
oPlatformCfg.numDeviceBlocks 1
db.numCameraModules 1
Pipeline: 3 ISP Output: 0 is using YUV 420 SEMI-PLANAR UINT8 BL REC_709ER
nvsipl_camera: Opened NITO file for module “HZKJ_IMX728_ES2_V2_120FOV”, file name: "/usr/share/camera/HZKJ_IMX728_ES2_V2_120FOVRetrieving metadata from NITO file for sensorID 3
Total number of knobsets in NITO file is 3
For Parameter Set:1
Parameter Set ID
86aab6ab-d0a7-894a-228af0f9d0da24e2
Parameter Set Schema Hash
8a99b5007f8010e85396cf26bc9130f10cd110245d1666355407c2739c66da8c
Parameter Set Data Hash
c13298f63d0ee7c3b1ee4922e61fa848c0835a3a736e0c3759949ae2153c55f2
For Parameter Set:2
Parameter Set ID
837fc831-4068-4344-929ba6fdffdcae99
Parameter Set Schema Hash
77e9209cded5f46d5e624d59a0d1b9c95bfd1b522961a49cb7b6dc76c9fa04f2
Parameter Set Data Hash
fb588f94a4326bf6bebe9fd25fa5da189baf75b4eb042a00efe8f58dd5d3ea0b
For Parameter Set:3
Parameter Set ID
8c2bcc48-96b6-2247-6292bec8cfb31d65
Parameter Set Schema Hash
e2e2254a72c1e3c7aa9a45eee3f4515df7b1151f65c23ba8b9c6acd428d6c93f
Parameter Set Data Hash
12946cd33eca812240f9434946a7173a686f56d312b3e6803c384a4bebf119f5

Enter ‘les’ followed by sensor ID to enable LED
Enter ‘lds’ followed by sensor ID to disable LED
Enter ‘dl’ followed by sensor ID to disable the link
Enter ‘el’ followed by sensor ID to enable the link without module reset
Enter ‘elr’ followed by sensor ID to enable the link with module reset
Enter ‘cm’ to check the module availabilty
Enter ‘q’ to quit the application

Output
Sensor3_Out0 Frame rate (fps): 28.999
Sensor3_Out1 Frame rate (fps): 28.999

10

@SivaRamaKrishnaNV - I spotted the mistake, the output-format in rig.json shall be only raw(RGGB) or processesd1(YUV420-SP) coming from ISP1.
“parameter”: “camera-name=HZKJ_IMX728_ES2_V2_120FOV,interface=csi-a,CPHY-mode=1,link=3,output-format=processed1”

Attaching the log, Kindly let me know if this is correct. I am runinng in headless mode and hence not able to visualize.
Thanks
dw_sample_camera_logs.txt (14.2 KB)

11

Dear @arkos,
Could you check using interface=csi-ab instead of csi-a. Also, could you try connecting camera to A0 port and use link=0.

Please see Mapping Connectors | NVIDIA Docs

12

@SivaRamaKrishnaNV : This is also running. Thanks
Attached the rig.json that works, for the above mentioned quanta camera.
I verified after a proper gdm3 uninstall/install and then vizualizing the camera output.

rig.json.txt (1.1 KB)

1 Like
13

Dear @arkos,
Glad to hear it is workikg. Can we close the topic?

14

Yes. many thanks

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.