How to record with the ar0144 gaze camera?

Hello,
I’m trying to use the GMSL camera AR0144 gaze on AGX Xavier with DRIVE 9.

I tried to access with the command :
./sample_camera_gmsl --camera-type=ar0144-cccc-none-gazet1
and
./sample_camera_gmsl_raw --camera-type=ar0144-cccc-none-gazet1

but I got errors :

./sample_camera_gmsl --camera-type=ar0144-cccc-none-gazet1
[04-11-2020 17:20:46] Platform: Detected DDPX - Tegra A
[04-11-2020 17:20:46] Initialize DriveWorks SDK v2.0.2081
[04-11-2020 17:20:46] Release build with GNU 4.9.4 from heads/buildbrain-branch-0-g36b127f against Drive PDK v5.1.0.2
[04-11-2020 17:20:46] TimeSource: monotonic epoch time offset is 1604477950357609
[04-11-2020 17:20:46] PTP Time is available from NVPPS Driver
[04-11-2020 17:20:46] Platform: number of GPU devices detected 1
[04-11-2020 17:20:46] Platform: currently selected GPU device integrated ID 0
[04-11-2020 17:20:46] SDK: Resources mounted from /usr/local/driveworks/data/
[04-11-2020 17:20:46] SDK: Create NvMediaDevice
[04-11-2020 17:20:46] SDK: use EGL display as provided
[04-11-2020 17:20:46] Initialize DriveWorks VisualizationSDK v2.0.2081
[04-11-2020 17:20:46] Initialize DriveWorksGL SDK v2.0.2081
[04-11-2020 17:20:46] SensorFactory::createSensor() -> camera.gmsl, output-format=yuv,fifo-size=3,isp-mode=yuv420-uint8,camera-type=ar0144-cccc-none-gazet1,camera-group=a,slave=0
[04-11-2020 17:20:46] Driveworks exception thrown: DW_INVALID_ARGUMENT: CameraGMSL: Sensor does not support requested output types

terminate called after throwing an instance of 'std::runtime_error'
  what():  [2020-11-04 17:20:46] DW Error DW_INVALID_ARGUMENT executing DW function:
 dwSAL_createSensor(&m_camera, params, m_sal)
 at /home/arnaud/dw_samples/src/sensors/camera/camera_gmsl/main.cpp:174
Aborted (core dumped)

and

./sample_camera_gmsl_raw --camera-type=ar0144-cccc-none-gazet1
[04-11-2020 17:54:44] Platform: Detected DDPX - Tegra A
[04-11-2020 17:54:44] Initialize DriveWorks SDK v2.0.2081
[04-11-2020 17:54:44] Release build with GNU 4.9.4 from heads/buildbrain-branch-0-g36b127f against Drive PDK v5.1.0.2
[04-11-2020 17:54:44] TimeSource: monotonic epoch time offset is 1604477950357609
[04-11-2020 17:54:44] PTP Time is available from NVPPS Driver
[04-11-2020 17:54:44] Platform: number of GPU devices detected 1
[04-11-2020 17:54:44] Platform: currently selected GPU device integrated ID 0
[04-11-2020 17:54:44] SDK: Resources mounted from /usr/local/driveworks-2.0/data/
[04-11-2020 17:54:44] SDK: Create NvMediaDevice
[04-11-2020 17:54:44] SDK: use EGL display as provided
[04-11-2020 17:54:44] Initialize DriveWorks VisualizationSDK v2.0.2081
[04-11-2020 17:54:44] Initialize DriveWorksGL SDK v2.0.2081
[04-11-2020 17:54:44] SensorFactory::createSensor() -> camera.gmsl, output-format=raw+data,camera-type=ar0144-cccc-none-gazet1,camera-group=a,format=raw,fifo-size=3,slave=0
[04-11-2020 17:54:44] CameraGMSL: Create NvMediaIPPManager
[04-11-2020 17:54:44] CameraGMSL: required FPS = 60, resolution = 1280x800
nvmedia: ERROR: ImgGetDevice: Can't find driver for ref_max96712_96705_ar0144
[04-11-2020 17:54:44] Driveworks exception thrown: DW_SAL_CANNOT_CREATE_SENSOR: CameraGMSL: cannot connect to camera

terminate called after throwing an instance of 'std::runtime_error'
  what():  [2020-11-04 17:54:44] DW Error DW_SAL_CANNOT_CREATE_SENSOR executing DW function:
 dwSAL_createSensor(&m_camera, params, m_sal)
 at /home/arnaud/dw_samples/src/sensors/camera/camera_gmsl_raw/main.cpp:178
Aborted (core dumped)

Do you have any idea how to solve this problem?

thanks,
Arnaud

Dear @delmotte,
Could you share camera module details? Please see the supported list in https://developer.nvidia.com/drive/ecosystem-hw-sw

Also, Can we know why you are on DRIVE SW 9.0? is it possible to upgrade to DRIVE SW 10.0?

Hello,
The camera model is ON Semiconductor AR0144ATSM-GEVK(GazeT-144).

We are on Drive SW 9 because most of the code developed by my company has been developed for this version and we did not upgraded yet. I can not update the machine I’m using because it is shared with other people. However, the seller of the module said it’s usable under drive 9.

After flashing tegra-b to put back a fresh install of drive 9, I could use sample_camera_gmsl_raw successfully but still not sample_camera_gmsl.
Is there any information on how to read the raw recorder files?
Is it directly the data from the sensor or is there any header?

Any idea how to make the processed mode work to record in h264?

I join here the new logs with a fresh tegra-b flash :

./sample_camera_gmsl --camera-type=ar0144-cccc-none-gazet1 --offscreen=1
WindowEGL: find EGL devices
WindowEGL: found 1 EGL devices
WindowEGL: init EGL with GLES3 context
WindowEGL: bind OpenGLES3 API
WindowEGL: offscreen mode -> use EGL PBuffer surface 1920x1080
WindowEGL: create EGL context
WindowEGL: assign EGL context to current thread
[09-11-2020 04:38:56] Platform: Detected DDPX - Tegra B
[09-11-2020 04:38:56] Initialize DriveWorks SDK v2.0.2081
[09-11-2020 04:38:56] Release build with GNU 4.9.4 from heads/buildbrain-branch-0-g36b127f against Drive PDK v5.1.0.2
[09-11-2020 04:38:56] TimeSource: monotonic epoch time offset is 1604896179285401
[09-11-2020 04:38:56] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[09-11-2020 04:38:56] PTP Time is available from Eth Driver
[09-11-2020 04:38:56] Platform: number of GPU devices detected 1
[09-11-2020 04:38:56] Platform: currently selected GPU device integrated ID 0
[09-11-2020 04:38:56] SDK: Resources mounted from /usr/local/driveworks/data/
[09-11-2020 04:38:56] SDK: Create NvMediaDevice
[09-11-2020 04:38:56] SDK: use EGL display as provided
[09-11-2020 04:38:56] Initialize DriveWorks VisualizationSDK v2.0.2081
[09-11-2020 04:38:56] Initialize DriveWorksGL SDK v2.0.2081
[09-11-2020 04:38:57] SensorFactory::createSensor() -> camera.gmsl, output-format=yuv,fifo-size=3,isp-mode=yuv420-uint8,camera-type=ar0144-cccc-none-gazet1,camera-group=a,slave=0
[09-11-2020 04:38:57] Driveworks exception thrown: DW_INVALID_ARGUMENT: CameraGMSL: Sensor does not support requested output types
terminate called after throwing an instance of 'std::runtime_error'
  what():  [2020-11-09 04:38:57] DW Error DW_INVALID_ARGUMENT executing DW function:
 dwSAL_createSensor(&m_camera, params, m_sal)
 at /home/arnaud/dw_samples/src/sensors/camera/camera_gmsl/main.cpp:174
Aborted (core dumped)

and

./sample_camera_gmsl_raw --camera-type=ar0144-cccc-none-gazet1 --offscreen=1
WindowEGL: find EGL devices
WindowEGL: found 1 EGL devices
WindowEGL: init EGL with GLES3 context
WindowEGL: bind OpenGLES3 API
WindowEGL: offscreen mode -> use EGL PBuffer surface 1280x800
WindowEGL: create EGL context
WindowEGL: assign EGL context to current thread
[09-11-2020 04:39:35] Platform: Detected DDPX - Tegra B
[09-11-2020 04:39:35] Initialize DriveWorks SDK v2.0.2081
[09-11-2020 04:39:35] Release build with GNU 4.9.4 from heads/buildbrain-branch-0-g36b127f against Drive PDK v5.1.0.2
[09-11-2020 04:39:35] TimeSource: monotonic epoch time offset is 1604896179285400
[09-11-2020 04:39:35] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[09-11-2020 04:39:35] PTP Time is available from Eth Driver
[09-11-2020 04:39:35] Platform: number of GPU devices detected 1
[09-11-2020 04:39:35] Platform: currently selected GPU device integrated ID 0
[09-11-2020 04:39:35] SDK: Resources mounted from /usr/local/driveworks-2.0/data/
[09-11-2020 04:39:35] SDK: Create NvMediaDevice
[09-11-2020 04:39:35] SDK: use EGL display as provided
[09-11-2020 04:39:35] Initialize DriveWorks VisualizationSDK v2.0.2081
[09-11-2020 04:39:35] Initialize DriveWorksGL SDK v2.0.2081
[09-11-2020 04:39:36] SensorFactory::createSensor() -> camera.gmsl, output-format=raw+data,camera-type=ar0144-cccc-none-gazet1,camera-group=a,format=raw,fifo-size=3,slave=0
[09-11-2020 04:39:36] CameraGMSL: Create NvMediaIPPManager
[09-11-2020 04:39:36] CameraGMSL: required FPS = 60, resolution = 1280x800
nvmedia isc: IscPwrCtrlInit: 939: Camera power control library: NVCCP
Max96712 Rev 2 detected!
MAX96712: Enable periodic AEQ on Link 0
MAX96705: Pre-emphasis set to 0xaa
Sensor AR0144 Rev 4.2 detected!
[09-11-2020 04:39:37] CameraGMSL: default IPP event callback function set
Successfully initialized camera with resolution of 1280x800 at framerate of 60.000000 FPS
[09-11-2020 04:39:37] CameraGMSL: Frame: 10 CaptureTimestamp: 1604896777837735 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 11 CaptureTimestamp: 1604896777854382 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 12 CaptureTimestamp: 1604896777871041 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 13 CaptureTimestamp: 1604896777887699 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 14 CaptureTimestamp: 1604896777904393 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 15 CaptureTimestamp: 1604896777921047 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 16 CaptureTimestamp: 1604896777937704 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 17 CaptureTimestamp: 1604896777954369 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 18 CaptureTimestamp: 1604896777971053 - ICP DROP in camera 0
[09-11-2020 04:39:37] CameraGMSL: Frame: 19 CaptureTimestamp: 1604896777987712 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 20 CaptureTimestamp: 1604896778004372 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 21 CaptureTimestamp: 1604896778021029 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 22 CaptureTimestamp: 1604896778037725 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 23 CaptureTimestamp: 1604896778054380 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 24 CaptureTimestamp: 1604896778071067 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 25 CaptureTimestamp: 1604896778087704 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 26 CaptureTimestamp: 1604896778104386 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 27 CaptureTimestamp: 1604896778121053 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 28 CaptureTimestamp: 1604896778137706 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 29 CaptureTimestamp: 1604896778154374 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 30 CaptureTimestamp: 1604896778171024 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 31 CaptureTimestamp: 1604896778187717 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 32 CaptureTimestamp: 1604896778204372 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 33 CaptureTimestamp: 1604896778221027 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 34 CaptureTimestamp: 1604896778237694 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 35 CaptureTimestamp: 1604896778254373 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 36 CaptureTimestamp: 1604896778271039 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 37 CaptureTimestamp: 1604896778287695 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 38 CaptureTimestamp: 1604896778304353 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 39 CaptureTimestamp: 1604896778321040 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 40 CaptureTimestamp: 1604896778337699 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 41 CaptureTimestamp: 1604896778354360 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 42 CaptureTimestamp: 1604896778371013 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 43 CaptureTimestamp: 1604896778387707 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 44 CaptureTimestamp: 1604896778404369 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 45 CaptureTimestamp: 1604896778421036 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 46 CaptureTimestamp: 1604896778437691 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 47 CaptureTimestamp: 1604896778454377 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 48 CaptureTimestamp: 1604896778471033 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 49 CaptureTimestamp: 1604896778487698 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 50 CaptureTimestamp: 1604896778504347 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 51 CaptureTimestamp: 1604896778521038 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 52 CaptureTimestamp: 1604896778537698 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 53 CaptureTimestamp: 1604896778554361 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 54 CaptureTimestamp: 1604896778571034 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 55 CaptureTimestamp: 1604896778587701 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 56 CaptureTimestamp: 1604896778604361 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 57 CaptureTimestamp: 1604896778621021 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 58 CaptureTimestamp: 1604896778637679 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 59 CaptureTimestamp: 1604896778654371 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 60 CaptureTimestamp: 1604896778671029 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 61 CaptureTimestamp: 1604896778687690 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 62 CaptureTimestamp: 1604896778704344 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 63 CaptureTimestamp: 1604896778721034 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 64 CaptureTimestamp: 1604896778737699 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 65 CaptureTimestamp: 1604896778754355 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 66 CaptureTimestamp: 1604896778771014 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 67 CaptureTimestamp: 1604896778787670 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 68 CaptureTimestamp: 1604896778804356 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 69 CaptureTimestamp: 1604896778821012 - ICP DROP in camera 0
[09-11-2020 04:39:38] CameraGMSL: Frame: 70 CaptureTimestamp: 1604896778837709 - ICP DROP in camera 0
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 0.000000
fps : 43.196545
fps : 40.167095
fps : 40.202621
fps : 40.017609
fps : 40.011204
fps : 37.523453
fps : 37.499531
fps : 35.314476
fps : 37.506565
fps : 37.506565
fps : 39.990402
fps : 39.990402
fps : 40.016006
fps : 42.830219
fps : 39.979210
fps : 39.990402
fps : 40.014404
fps : 39.996799
fps : 39.998402
fps : 39.998402
fps : 40.016006
fps : 42.846737
fps : 39.996799
fps : 37.519230
fps : 37.523453
fps : 37.498127
fps : 37.498127
fps : 37.503750
fps : 37.509377
fps : 37.496719
fps : 35.285816
fps : 33.344448
fps : 33.332222
fps : 33.338890
fps : 35.294533
fps : 35.305748
fps : 35.303253
fps : 37.495312
fps : 35.305748
fps : 37.509377
fps : 40.014404
fps : 42.844902
fps : 42.866940
fps : 42.854080
fps : 40.004799
fps : 42.844902
fps : 40.009602
fps : 37.506565
fps : 39.980808
fps : 37.499531
fps : 35.293285
fps : 33.340000
fps : 35.289551
fps : 35.302010
fps : 35.304501
fps : 35.295780
fps : 35.295780
fps : 37.505157
fps : 37.509377
fps : 37.502342
fps : 37.506565
fps : 40.003201
fps : 37.522045
fps : 39.995201
fps : 39.996799
fps : 40.011204
fps : 39.995201
fps : 37.509377
fps : 37.506565
fps : 37.507969
fps : 40.004799
fps : 40.003201
fps : 40.001598
fps : 39.995201
fps : 42.833889
fps : 40.001598
fps : 42.874294
fps : 46.123333
fps : 42.848572
fps : 46.125462
fps : 46.127590
fps : 42.861427
fps : 42.874294
fps : 42.863266
fps : 42.872456
fps : 42.861427
fps : 42.844902
fps : 40.016006
fps : 42.861427
fps : 40.024014
fps : 40.014404
fps : 40.020809
fps : 42.835724
fps : 42.855919
fps : 40.003201
fps : 39.998402
fps : 37.500938
fps : 37.505157
^CTiming results:
Thread main:
-onProcess                CPU: 15831us, std=1546       | GPU: 15827us, std=1560       | samples=167
-onRender                 CPU:    74us, std=  17       | GPU:    72us, std=  14       | samples=108
[09-11-2020 04:39:41] Sensor statistics for: camera.gmsl, output-format=raw+data,camera-type=ar0144-cccc-none-gazet1,camera-group=a,format=raw,fifo-size=3,slave=0
[09-11-2020 04:39:41] Events: 169
Errors: 0
Drops: 61
minDelta: 16615.0000000
maxDelta: 1033311.0000000
meanDelta: 22753.562500
Standard deviation: 78434.476563
[09-11-2020 04:39:41] Driveworks VisualizationSDK released
[09-11-2020 04:39:41] SDK: Release NvMediaDevice
[09-11-2020 04:39:41] DriveworksGL SDK released
[09-11-2020 04:39:41] Driveworks SDK released
[09-11-2020 04:39:41] SDK: Release NvMedia2D

thanks,
Arnaud

Dear @delmotte,
The camera support only RAW format. So only sample_camera_gmsl_raw works.
RAW data corresponds to data coming from sensor.
You can check sample_camera_replay code to know more details on reading raw video.

Thanks for the support, I succeeded to visualize the recorded video using sample_camera_replay.

If I want to record it in h264, is there a way to process it using dwSoftISP and stream the result to the h264 serializer of driveworks?
Or should I stream the rgba data to the cpu and then use NvMedia API for the encoding like for USB cameras?

thanks,
Arnaud

Dear @delmotte,
Could you check Modules -> SAL -> ISP section for your topic? Please file a new topic for new issues unrelated to thread to avoid cluttering of topics.