Recorder application failing

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
[*] DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
[*] Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
[*] NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
[*] 1.7.1.8928
other

Host Machine Version
[*] native Ubuntu 18.04
other

List item
When I’m running the recorder application as below, I’m facing an error.
root@tegra-ubuntu:/usr/local/driveworks-3.5/tools/capture# ./recorder /usr/local/driveworks-3.5/data/samples/sensors/camera/camera/rig.json

Error:

[18-02-2022 12:26:16] CameraMaster: master initiation

ssDEVBLK_WORKER_0: pwr_utils.cpp: 196: PowerControlRequestOwnership: PowerControlRequestOwnership: nvccp_request_ownership failed with status (1005)

DEVBLK_WORKER_0: CNvMDeviceBlockMgr.hpp: 147: ThreadFunc: Device block init failed(7f)

recorder: CNvMDeviceBlockMgr.cpp: 129: WorkersDoAction: Device block action:1 failed for block:0

recorder: CNvMDeviceBlockMgr.cpp: 163: Init: Failed to initialize device blocks

recorder: CNvMCamera.cpp: 436: Init: DeviceBlock manager init failed

[18-02-2022 12:26:21] CameraMaster: master init failed127

recorder: CNvMCamera.cpp: 568: GetImageAttributes: Incorrect pipeline state:1 for pipeline:0

[18-02-2022 12:26:21] CameraMaster: GetImageAttributes failed6

[18-02-2022 12:26:21] CameraClient: GetImageAttributes failed6

[18-02-2022 12:26:21] CameraClient: Failed to allocate images groups!6

terminate called after throwing an instance of 'dw::core::Exception'

what(): DW_FAILURE: CameraClient: Failed to allocate images pool!

Aborted (core dumped)

Two Sekonix camera were running the sample_camera application:

Even with recorder-tui application we are getting this below error :

running recorder-tui with arguments: Namespace(aes_key=None, aes_key_encrypted=None, backend=False, bbr=False, disable_encryption=True, enable_encryption=False, no_mount_check=False, non_interaction=False, remote_dw_path='', rig='/usr/local/driveworks-3.5/data/samples/sensors/camera/camera/rig.json', rsa_key='/home/nvidia/.ssh/recorder-aiinfra.pem', rsa_key_md5=None, skip_init_input=False, tag='NONE')

[18-02-2022 13:03:03] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.

[18-02-2022 13:03:03] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.

[18-02-2022 13:03:04] SDK: No resources(.pak) mounted from '/usr/local/driveworks-3.5/data'. Please adjust path or some modules won't function properly.

[18-02-2022 13:03:04] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.

[18-02-2022 13:03:04] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.

Copying rigs and setup code for local...

Platform setup for local...

Backend recorder didn't start, no acknowledgement received

Errors detected during recording

First 100 lines of std error log:

First 100 lines of std out log:

When reporting bugs, please attach full log and files /tmp/recorder-tui-12975/*.log and /tmp/recorder-12975/*.log

WARNING: Forcibly exited. Some data might have been lost!

WARNING: Use q<Enter> to exit gracefully, next time.

Complete error stack trace is given below:

root@tegra-ubuntu:/usr/local/driveworks-3.5/tools/capture# ./recorder /usr/local/driveworks-3.5/data/samples/sensors/camera/camera/rig.json
[18-02-2022 12:26:14] Platform: Detected DDPX - Tegra A
[18-02-2022 12:26:14] TimeSource: monotonic epoch time offset is 1644926220453768
[18-02-2022 12:26:14] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.
[18-02-2022 12:26:14] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[18-02-2022 12:26:14] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.
[18-02-2022 12:26:14] TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
[18-02-2022 12:26:14] Platform: number of GPU devices detected 2
[18-02-2022 12:26:14] Platform: currently selected GPU device discrete ID 0
[18-02-2022 12:26:14] Context::getDataPathFromSelfLocation DATA_ROOT found at: /usr/local/driveworks-3.5/data
[18-02-2022 12:26:14] SDK: No resources(.pak) mounted, some modules will not function properly
[18-02-2022 12:26:14] SDK: Create NvMediaDevice
[18-02-2022 12:26:14] SDK: Create NvMedia2D
[18-02-2022 12:26:14] egl::Display: found 2 EGL devices
[18-02-2022 12:26:14] egl::Display: use drm device: /dev/dri/card0
[18-02-2022 12:26:14] TimeSource: monotonic epoch time offset is 1644926220453768
[18-02-2022 12:26:14] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.
[18-02-2022 12:26:14] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[18-02-2022 12:26:14] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.
[18-02-2022 12:26:14] TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
[18-02-2022 12:26:14] Initialize DriveWorks SDK v3.5.75
[18-02-2022 12:26:14] Release build with GNU 7.3.1 from heads/buildbrain-branch-0-gc61a9a35bd0 against Drive PDK v5.2.0.0
[18-02-2022 12:26:14] Platform: currently selected GPU device integrated ID 1
[18-02-2022 12:26:14] Rig::fromFile: Loading rig file: /usr/local/driveworks-3.5/data/samples/sensors/camera/camera/rig.json
[18-02-2022 12:26:14] No valid data file found for camera:front:center:120fov in parameter string: camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=0,format=h264,output-format=yuv (using configuration folder /usr/local/driveworks-3.5/data/samples/sensors/camera/camera/)
[18-02-2022 12:26:14] No valid data file found for camera:front:right:120fov in parameter string: camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=1,format=h264,output-format=yuv (using configuration folder /usr/local/driveworks-3.5/data/samples/sensors/camera/camera/)
[18-02-2022 12:26:14] SensorFactory::createSensor() -> camera.gmsl, camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=0,format=h264,output-format=yuv
[18-02-2022 12:26:14] CameraGMSLSelector: cameraCount is a deprecated parameter, please remove as it has no more effect
[18-02-2022 12:26:14] CameraGMSLSelector: creating CameraClient from remapped old gmsl parameters to camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=0,format=h264,output-format=yuv,camera-name=SF3324,interface=csi-ab,link=0
[18-02-2022 12:26:14] CameraBase: pool size set to 8
[18-02-2022 12:26:14] SensorFactory::createSensor() -> camera.gmsl.master,
[18-02-2022 12:26:14] CameraMaster::parseDevBlock Getting device info list.
[18-02-2022 12:26:14] devBlock: 1 Slave = 0 Interface = csi-ab Camera_name = SF3324 Link = 0
[18-02-2022 12:26:14] Camera Match Name: SF3324 Description: Sekonix SF3324 module - 120-deg FOV, DVP AR0231-RCCB, MAX96705 linkIndex: 4294967295 serInfo.Name: MAX96705
[18-02-2022 12:26:14] Client, Setting up information for camera ID 0
[18-02-2022 12:26:14] Client, successfully found info for camera ID 0 bound to id 0
[18-02-2022 12:26:15] CameraClient: no NITO found at /opt/nvidia/nvmedia/nit/SF3324.nito
[18-02-2022 12:26:15] CameraClient: using NITO found at /opt/nvidia/nvmedia/nit/sf3324.nito
[18-02-2022 12:26:15] SensorFactory::createSensor() -> camera.gmsl, camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=1,format=h264,output-format=yuv
[18-02-2022 12:26:15] CameraGMSLSelector: cameraCount is a deprecated parameter, please remove as it has no more effect
[18-02-2022 12:26:15] CameraGMSLSelector: creating CameraClient from remapped old gmsl parameters to camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=1,format=h264,output-format=yuv,camera-name=SF3324,interface=csi-ab,link=1
[18-02-2022 12:26:15] CameraBase: pool size set to 8
[18-02-2022 12:26:15] devBlock: 1 Slave = 0 Interface = csi-ab Camera_name = SF3324 Link = 1
[18-02-2022 12:26:15] Camera Match Name: SF3324 Description: Sekonix SF3324 module - 120-deg FOV, DVP AR0231-RCCB, MAX96705 linkIndex: 4294967295 serInfo.Name: MAX96705
[18-02-2022 12:26:15] Client, Setting up information for camera ID 1
[18-02-2022 12:26:15] Client, successfully found info for camera ID 1 bound to id 1
[18-02-2022 12:26:16] CameraClient: no NITO found at /opt/nvidia/nvmedia/nit/SF3324.nito
[18-02-2022 12:26:16] CameraClient: using NITO found at /opt/nvidia/nvmedia/nit/sf3324.nito
[18-02-2022 12:26:16] CameraClient: camera params: index_table=off,file=,camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=0,format=h264,output-format=yuv
[18-02-2022 12:26:16] CameraClient: serializer bitrate not specified. Using 8000000.
[18-02-2022 12:26:16] EncoderNvMedia: Setting encode on instance 0
[18-02-2022 12:26:16] CameraClient: camera params: index_table=off,file=,camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=2,siblingIndex=1,format=h264,output-format=yuv
[18-02-2022 12:26:16] CameraClient: serializer bitrate not specified. Using 8000000.
[18-02-2022 12:26:16] EncoderNvMedia: Setting encode on instance 0
Rig: rig.json NewSink: /dev/null
[18-02-2022 12:26:16] CameraGSMLMaster: starting...
[18-02-2022 12:26:16] SIPLMaster::SIPLMaster: Setting up master camera
[18-02-2022 12:26:16] Platform:
[18-02-2022 12:26:16] Platform Config:
[18-02-2022 12:26:16] Description:
[18-02-2022 12:26:16] Number of device blocks: 1
[18-02-2022 12:26:16] Device Block : 0
[18-02-2022 12:26:16]     csiPort: 2
[18-02-2022 12:26:16]     i2cDevice: 0
[18-02-2022 12:26:16]     Deserializer Name: MAX96712
[18-02-2022 12:26:16]     Deserializer Description: Maxim 96712 Aggregator
[18-02-2022 12:26:16]     Deserializer i2cAddress: 41
[18-02-2022 12:26:16]     Simulator Mode: 0
[18-02-2022 12:26:16]     Slave Mode: 0
[18-02-2022 12:26:16]     Phy Mode: 0
[18-02-2022 12:26:16]     Number of camera modules: 2
[18-02-2022 12:26:16]     CameraModule index: 0
[18-02-2022 12:26:16]    	 Name :SF3324
[18-02-2022 12:26:16]    	 Description: Sekonix SF3324 module - 120-deg FOV, DVP AR0231-RCCB, MAX96705
[18-02-2022 12:26:16]    	 Serializer name: MAX96705
[18-02-2022 12:26:16]    	 Serializer description: Maxim 96705 Serializer
[18-02-2022 12:26:16]    	 Serializer i2cAdress: 64
[18-02-2022 12:26:16]    		 Sensor ID: 0
[18-02-2022 12:26:16]    		 Sensor name: AR0231
[18-02-2022 12:26:16]    		 Sensor description: OnSemi AR0231 Sensor
[18-02-2022 12:26:16]    		 Sensor i2cAddress: 16
[18-02-2022 12:26:16]    		 Sensor isTPGEnabled: 0
[18-02-2022 12:26:16]    		 Sensor isTriggerMode: 1
[18-02-2022 12:26:16]    			  cfa: 39
[18-02-2022 12:26:16]    			  embeddedTopLines: 24
[18-02-2022 12:26:16]    			  embeddedBottomLines: 4
[18-02-2022 12:26:16]    			  inputFormat: 8
[18-02-2022 12:26:16]    			  height: 1208
[18-02-2022 12:26:16]    			  width: 1920
[18-02-2022 12:26:16]    			  fps: 30.0000000
[18-02-2022 12:26:16]    			  Embedded Data: 0
[18-02-2022 12:26:16]     CameraModule index: 1
[18-02-2022 12:26:16]    	 Name :SF3324
[18-02-2022 12:26:16]    	 Description: Sekonix SF3324 module - 120-deg FOV, DVP AR0231-RCCB, MAX96705
[18-02-2022 12:26:16]    	 Serializer name: MAX96705
[18-02-2022 12:26:16]    	 Serializer description: Maxim 96705 Serializer
[18-02-2022 12:26:16]    	 Serializer i2cAdress: 64
[18-02-2022 12:26:16]    		 Sensor ID: 1
[18-02-2022 12:26:16]    		 Sensor name: AR0231
[18-02-2022 12:26:16]    		 Sensor description: OnSemi AR0231 Sensor
[18-02-2022 12:26:16]    		 Sensor i2cAddress: 16
[18-02-2022 12:26:16]    		 Sensor isTPGEnabled: 0
[18-02-2022 12:26:16]    		 Sensor isTriggerMode: 1
[18-02-2022 12:26:16]    			  cfa: 39
[18-02-2022 12:26:16]    			  embeddedTopLines: 24
[18-02-2022 12:26:16]    			  embeddedBottomLines: 4
[18-02-2022 12:26:16]    			  inputFormat: 8
[18-02-2022 12:26:16]    			  height: 1208
[18-02-2022 12:26:16]    			  width: 1920
[18-02-2022 12:26:16]    			  fps: 30.0000000
[18-02-2022 12:26:16]    			  Embedded Data: 0
[18-02-2022 12:26:16] CameraMaster::setOutputDescription Setting output consumer descriptors for sensor:  OutputType: 1
[18-02-2022 12:26:16] CameraMaster::setOutputDescription Setting output consumer descriptors for sensor:  OutputType: 1
[18-02-2022 12:26:16] Client, setting pipeline config for camera ID 0
[18-02-2022 12:26:16] Client, setting pipeline config for camera ID 1
[18-02-2022 12:26:16] CameraMaster: master initiation
ssDEVBLK_WORKER_0: pwr_utils.cpp: 196: PowerControlRequestOwnership: PowerControlRequestOwnership: nvccp_request_ownership failed with status (1005)
DEVBLK_WORKER_0: CNvMDeviceBlockMgr.hpp: 147: ThreadFunc: Device block init failed(7f)
recorder: CNvMDeviceBlockMgr.cpp: 129: WorkersDoAction: Device block action:1 failed for block:0
recorder: CNvMDeviceBlockMgr.cpp: 163: Init: Failed to initialize device blocks
recorder: CNvMCamera.cpp: 436: Init: DeviceBlock manager init failed
[18-02-2022 12:26:21] CameraMaster: master init failed127
recorder: CNvMCamera.cpp: 568: GetImageAttributes: Incorrect pipeline state:1 for pipeline:0
[18-02-2022 12:26:21] CameraMaster: GetImageAttributes failed6
[18-02-2022 12:26:21] CameraClient: GetImageAttributes failed6
[18-02-2022 12:26:21] CameraClient: Failed to allocate images groups!6
terminate called after throwing an instance of 'dw::core::Exception'
  what():  DW_FAILURE: CameraClient: Failed to allocate images pool!
Aborted (core dumped)

Dear @raji,
As I understand, you can run sample_camera application with a rig file and failed to record using same rig file. If so, could you please share the rig file for reproducing at our end?
Please make sure the cameras are not used by any other application and re-check after restarting the target.

Here I have attached the rig config file -
cameraRig.txt (1.9 KB)

Dear @raji,
as discussed offline, you need to run only recorder alone and no need to run sample_camera application in parallel.

Please file a new topic if you have any other issue not relavant to this thread

Could you also please check and respond to reason recorder-tui application failure?

Dear @raji,
Please confirm if recorder app is able to record using Json file. If so, please file a new topic with all steps to reproduce the issue with recorder-tui.

Also, We recommend posting text message logs instead of pictures to be able to search by community users.

okay sure thank you.