Camera frame serialization with Driveworks api performance optimization

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
[*] DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[*] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
[*] other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[*] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Dear NVIDIA Teams,

We have deployed Driveworks in our project and we are trying to serialize camera raw frame to h264 format with Driveworks api dwSensorSerializer_serializeCameraFrameAsync. After couple of trials with different camera , we found out that this api takes around 4ms to serialize one 2MP camera raw frame and 16ms to setialize one 8MP camera raw frame. While we are running multiple 8MP and 2MP cameras and serialize all of them consecutively, the serialization process takes quite long time which results in low frame rate of camera output.

We are following the exactly same Driveworks camera sample code structure in our project, and we are wondering if there there are any way to optimize the serialization process to improve the overall frame rate of cameras.

Thanks,

3

May I know how did you measure it? Did you try using our recorder tool? Is there any reason for using DRIVE OS 6.0.4 release? If possible could you move to new release(DRIVE OS 6.0.8.1)?

4

We are using customized cameras, and our camera driver doesn’t support DRIVE OS 6.0.8.1 yet so we have to stick with DRIVE OS 6.0.4 for a while. While creating the dwSensorSerializerHandle_t, we add a callback function to the serializer by passing our customized function to dwSerializerParams.onData. We measured the timestamp difference between the moment right before calling the function dwSensorSerializer_serializeCameraFrameAsync and the moment once get into our callback function, and the results are around 4ms for 2MP camera raw frame and 16ms for 8MP camera raw frame in our case. We haven’t tried the recorder tool that you mentioned yet, is that the sample_record in the driveworks or the other one in Nvmedia?
Thanks!

5

Since our camera driver doesn’t support DRIVE OS 6.0.8.1 yet, so we were wondering

  1. With Driveworks on DRIVE OS 6.0.8.1, how many 8MP cameras raw frames can be simultaneously serialized together without significant frame loss(below 30fps)?
  2. To optimize the serialization process, we have tried to resize the camera raw frame with Driveworks Api dwImageTransformation_copyFullImage from 8mp to 2mp, and realize that the resized camera frame can’t be serialized anymore as Driveworks Api dwSensorSerializer_serializeCameraFrameAsync is not compatible with the serialization of processed frame. So is there any way to serialize the camera frame after resized it with only Driveworks API?

Thanks

6

Recorder tool is at DriveWorks SDK Reference: Basic Recording Tool . Using 30FPS, if you see framedrop issues, please check using frame-mod (see DriveWorks SDK Reference: Configuration Reference).
Could you check write disk speed of storage location with dd command?

7

Hello,
I tried the similar testing on the driveos6.0.8(Driveworks 5.14) and the result is even worse it took about 28~29 ms to compress an 8MP image.
I modified sample_camera to make * * dwSerializerParams. onData * * work,also measured the timestamp difference between the moment right before calling the function dwSensorSerializer_serializeCameraFrameAsync and the moment once get into our callback function.
Do I need to set other parameters to improve compression performance？

Camera: AR0820 8MP

Log:
Warning: SamplesDataPath::getBasePathFor(): ‘DATA_ROOT’ not found in search paths [/home/nvidia/samples_bin/data | /home/nvidia/samples_bin.runfiles/av/data | /home/nvidia/samples_bin/sample_camera.runfiles/av/data | /home/nvidia/samples_bin/data | /home/nvidia/data | /home/data]
ProgramArguments: Missing argument ‘dwTracePath’ requested
WindowEGL: find EGL devices
WindowEGL: found 1 EGL devices
WindowEGL: init EGL with GLES3 context
WindowEGL: bind OpenGLES3 API
WindowEGL: offscreen mode → use EGL PBuffer surface 1280x800
WindowEGL: create EGL context
WindowEGL: assign EGL context to current thread
[25-03-2024 10:46:12] Platform: Detected Drive Orin P3710
[25-03-2024 10:46:12] Adding variable DW_Base:DW_Version
[25-03-2024 10:46:12] Added variable DW_Base:DW_Version
[25-03-2024 10:46:12] Platform: number of GPU devices detected 1
[25-03-2024 10:46:12] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_10_03, Arch: ga10b
[25-03-2024 10:46:12] Platform: currently selected GPU device integrated ID 0
[25-03-2024 10:46:12] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[25-03-2024 10:46:12] CUDLAEngine:getDLACount: Number of DLA devices = 2
[25-03-2024 10:46:12] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘./resources’: VirtualFileSystem: Failed to mount ‘./resources/resources.pak’
[25-03-2024 10:46:12] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/home/nvidia/samples_bin/data’: VirtualFileSystem: Failed to mount ‘/home/nvidia/samples_bin/data/resources.pak’
[25-03-2024 10:46:12] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.14/data
[25-03-2024 10:46:12] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.14/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.14/data/resources.pak’
[25-03-2024 10:46:12] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.14/data
[25-03-2024 10:46:12] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.14/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.14/data/resources.pak’
[25-03-2024 10:46:12] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.14/targets/aarch64-Linux/lib/libdw_base.so.5.14
[25-03-2024 10:46:12] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.14/targets/aarch64-Linux/lib/libdw_base.so.5.14
[25-03-2024 10:46:12] SDK: No resources(.pak) mounted, some modules will not function properly
[25-03-2024 10:46:12] SDK: use EGL display as provided
[25-03-2024 10:46:12] [25-03-2024 10:46:12] Initialize DriveWorks SDK v5.14.74
[25-03-2024 10:46:12] [25-03-2024 10:46:12] Release build with GNU 9.3.0 from buildbrain-branch-0-g30b7ba5bb0c against Drive PDK v6.0.8.0
[25-03-2024 10:46:12] SensorFactory::createSensor() → time.nvpps, nvpps-device=/dev/nvpps0
[25-03-2024 10:46:12] Found driver: time.nvpps
[25-03-2024 10:46:12] Can not create time sensor: DW_HAL_CANNOT_OPEN_CHANNEL: EndpointNVPPS: cannot open NVPPS device with name: /dev/nvpps0
Error: Permission denied
[25-03-2024 10:46:12] fromFile: Loading rig file: ./rig_ar0820.json
[25-03-2024 10:46:12] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
[25-03-2024 10:46:12] No valid data file found for camera:sample0 in parameter string: camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216 (using configuration folder ./)
[25-03-2024 10:46:12] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
onInitialize: creating camera.gmsl with params: camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:12] SensorFactory::createSensor() → camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:12] Found driver: camera.gmsl
[25-03-2024 10:46:12] CameraBase: pool size set to 8
[25-03-2024 10:46:12] SensorFactory::createSensor() → camera.gmsl.master,
[25-03-2024 10:46:12] Found driver: camera.gmsl.master
[25-03-2024 10:46:13] CameraMaster::parseDevBlock Getting device info list.
[25-03-2024 10:46:13] devBlock: 0 Slave = 0 Interface = csi-ef Camera_name = F008A120RM0AV2 Link = 0 GroupInit = 0
[25-03-2024 10:46:13] Camera Match Name: F008A120RM0AV2 Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295 linkIndex: 4294967295 serInfo.Name: MAX9295
[25-03-2024 10:46:13] Authentication is disabled for camera F008A120RM0AV2
[25-03-2024 10:46:13] Not an EEPROM supported camera F008A120RM0AV2
[25-03-2024 10:46:13] Not an EEPROM supported camera F008A120RM0AV2
[25-03-2024 10:46:13] Client, Setting up information for camera ID 0
[25-03-2024 10:46:13] Client, successfully found info for camera ID 0 bound to id 0
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] ImageStreamer(NvMedia → CUDA)
[25-03-2024 10:46:13] CameraClient: using NITO found at /usr/share/camera/F008A120RM0AV2.nito
[25-03-2024 10:46:13] CameraClient: CVARS not registered for camera: F008A120RM0AV2
[25-03-2024 10:46:13] SyncDriver: GPU device available, enabling CUDA sync
[25-03-2024 10:46:13] SyncDriver: enableCUDAPresync, object successfully created
[25-03-2024 10:46:13] SIPLMaster::SIPLMaster: Setting up master camera
[25-03-2024 10:46:13] Platform:
[25-03-2024 10:46:13] Platform Config:
[25-03-2024 10:46:13] Description:
[25-03-2024 10:46:13] Number of device blocks: 1
[25-03-2024 10:46:13] Device Block : 0
[25-03-2024 10:46:13] csiPort: 8
[25-03-2024 10:46:13] i2cDevice: 2
[25-03-2024 10:46:13] cphyRate_0: 2000000
[25-03-2024 10:46:13] cphyRate_1: 2000000
[25-03-2024 10:46:13] Deserializer Name: MAX96712
[25-03-2024 10:46:13] Deserializer Description: Maxim 96712 Aggregator
[25-03-2024 10:46:13] Deserializer i2cAddress: 41
[25-03-2024 10:46:13] Passive Mode: 0
[25-03-2024 10:46:13] Phy Mode: 1
[25-03-2024 10:46:13] Number of camera modules: 1
[25-03-2024 10:46:13] CameraModule index: 0
[25-03-2024 10:46:13] Name :F008A120RM0AV2
[25-03-2024 10:46:13] Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295
[25-03-2024 10:46:13] Serializer name: MAX9295
[25-03-2024 10:46:13] Serializer description: Maxim 9295 Serializer
[25-03-2024 10:46:13] Serializer i2cAdress: 98
[25-03-2024 10:46:13] EEPROM name: N24C64
[25-03-2024 10:46:13] EEPROM i2cAddress: 84
[25-03-2024 10:46:13] EEPROM description: N24C64 EEPROM
[25-03-2024 10:46:13] Sensor ID: 0
[25-03-2024 10:46:13] Sensor name: AR0820
[25-03-2024 10:46:13] Sensor description: OnSemi AR0820 Sensor
[25-03-2024 10:46:13] Sensor isTPGEnabled: 0
[25-03-2024 10:46:13] Sensor i2cAddress: 16
[25-03-2024 10:46:13] Sensor isTriggerMode: 1
[25-03-2024 10:46:13] cfa: 35
[25-03-2024 10:46:13] embeddedTopLines: 2
[25-03-2024 10:46:13] embeddedBottomLines: 4
[25-03-2024 10:46:13] inputFormat: 8
[25-03-2024 10:46:13] height: 2168
[25-03-2024 10:46:13] width: 3848
[25-03-2024 10:46:13] fps: 30.000000
[25-03-2024 10:46:13] Embedded Data: 0
[25-03-2024 10:46:13] CameraMaster available outputs: 1
[25-03-2024 10:46:13] CameraClient: setting pipeline config for camera ID 0
[25-03-2024 10:46:13] CameraClient, SIPLNotifier created
[25-03-2024 10:46:13] CameraMaster: master initiation
MAX96712: Revision 5 detected
MAX96712 Link 0: PHY optimization was enabled
MAX9295: Revision 8 detected!
Sensor AR0820 GRBG Rev 2.1 detected!
[25-03-2024 10:46:13] CameraMaster: Bootstrap NvSIPL and ClientPipelines complete
[25-03-2024 10:46:13] SyncDriver: GPU device available, enabling CUDA sync
[25-03-2024 10:46:13] SyncDriver: enableCUDAPresync, object successfully created
[25-03-2024 10:46:13] CameraClient::readEEPROMInfo:
[25-03-2024 10:46:13] Not an EEPROM supported camera F008A120RM0AV2
[25-03-2024 10:46:13] mmIntrinsicModelID: 27
[25-03-2024 10:46:13] mmIntrinsicCoeff[0]: -0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[1]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[2]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[3]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[4]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[5]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[6]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[7]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[8]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[9]: 1.892143e+11
[25-03-2024 10:46:13] mmIntrinsicCoeff[10]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[11]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[12]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[13]: 0.000000
[25-03-2024 10:46:13] mmIntrinsicCoeff[14]: 0.000170
[25-03-2024 10:46:13] mmIntrinsicCoeff[15]: 8.000000
[25-03-2024 10:46:13] mmIntrinsicMeasuredFlagID: 237
[25-03-2024 10:46:13] Not an EEPROM supported camera F008A120RM0AV2
[25-03-2024 10:46:13] fwVersionMajor: 0
[25-03-2024 10:46:13] fwVersionMinor: 0
[25-03-2024 10:46:13] fwVersionPatch: 0
[25-03-2024 10:46:13] CameraClient: format not specified. Using mp4 in Codec Header.
[25-03-2024 10:46:13] [25-03-2024 10:46:13] SAL: bootstrap complete
[25-03-2024 10:46:13] Initialize DriveWorks VisualizationSDK v5.14.74
[25-03-2024 10:46:13] Initialize DriveWorksGL SDK v5.14.74
onInitialize: Total cameras 1
onInitialize: getting image props 0
onInitialize: initilizing stream: 0
----- format(h264=0) = 0
---- 111 the old ser parameters : camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216,type=disk,file=./ar0820_out.h264
,used param:camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,format=h264,framerate=30,bitrate=32000000,type=user
[25-03-2024 10:46:13] CameraClient: camera params: camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,format=h264,framerate=30,bitrate=32000000,type=user
[25-03-2024 10:46:13] CameraClient: h264 and h265 video format encoding is deprecated. Please switch to a different format.
[25-03-2024 10:46:13] CameraClient::createSerializer: Serializer created for ISP Pipeline: 0
[25-03-2024 10:46:13] SyncDriver: GPU device available, enabling CUDA sync
[25-03-2024 10:46:13] SyncDriver: enableCUDAPresync, object successfully created
[25-03-2024 10:46:13] EncoderNvMedia: Setting encode on instance 0
[25-03-2024 10:46:13] EncoderNvMedia: Header Information not available for h264/h265 , Default Header Data
[25-03-2024 10:46:13] SyncDriver: GPU device available, enabling CUDA sync
[25-03-2024 10:46:13] SyncDriver: enableCUDAPresync, object successfully created
@@@@@ initializeSerializer 0! status=0
Main: Starting master.
[25-03-2024 10:46:13] CameraMaster: starting…
[25-03-2024 10:46:13] CameraClient: allocating image pool for ICP
[25-03-2024 10:46:13] Allocating NvSci group
[25-03-2024 10:46:13] CameraClient: allocate image pool for 0
[25-03-2024 10:46:13] Allocating NvSci ISP0 images
[25-03-2024 10:46:13] SIPLClient: Acquisition started
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363573839.713
###> After serializerOnDataFunc, datasize: 17084,endTimePoint: 1711363573868.837**,elpsedTime: 29.123**
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363573917.001
###> After serializerOnDataFunc, datasize: 250688,endTimePoint: 1711363573946.645,elpsedTime: 29.644
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363573962.166
###> After serializerOnDataFunc, datasize: 273652,endTimePoint: 1711363573991.538,elpsedTime: 29.372
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363574004.539
###> After serializerOnDataFunc, datasize: 312080,endTimePoint: 1711363574033.797,elpsedTime: 29.258
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363574046.739
###> After serializerOnDataFunc, datasize: 313103,endTimePoint: 1711363574075.836,elpsedTime: 29.097
===> before_calling_dwSensorSerializer_serializeCameraFrameAsync timepoint:1711363574088.769
[25-03-2024 10:46:14] SIPLNotifier: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
…
[25-03-2024 10:46:15] SIPLDeviceBlockNotificationHandler: Queue timeout
Timing results:
Thread main:
-onProcess CPU: 4us, std= 5 | GPU: 4us, std= 1 | samples=45
-onRender CPU: 41246us, std=19502 | GPU: 41271us, std=19497 | samples=45

[25-03-2024 10:46:15] Sensor Health Signal for camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
timestamp: 1711363575773184 sourceID: 1
Sensor notifications:
[25-03-2024 10:46:15] Sensor statistics for: camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:15] Statistics for DW_SENSOR_STATE_DELTA_HOST_AND_HOST_TIME
Events: 94
Errors: 0
CRC Errors: 0
Drops: 10
minDelta: 33292.0000 us
maxDelta: 66701.0000 us
meanDelta: 40152.2969us
Standard deviation: 13443.1006
[25-03-2024 10:46:15] Sensor statistics for: camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:15] Statistics for DW_SENSOR_STATE_DELTA_SENSOR_AND_SENSOR_TIME
Events: 94
Errors: 0
CRC Errors: 0
Drops: 10
minDelta: 0.0000 us
maxDelta: 0.0000 us
meanDelta: 0.0000us
Standard deviation: 0.0000
[25-03-2024 10:46:15] Sensor statistics for: camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:15] Statistics for DW_SENSOR_STATE_DELTA_HOST_AND_SENSOR_TIME
Events: 94
Errors: 0
CRC Errors: 0
Drops: 10
minDelta: -2.8130e+14 us
maxDelta: -2.8130e+14 us
meanDelta: -2.8130e+14us
Standard deviation: 0.0000
[25-03-2024 10:46:15] Sensor statistics for: camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:15] Statistics for DW_SENSOR_STATE_DELTA_CURRENT_AND_HOST_TIME
Events: 94
Errors: 0
CRC Errors: 0
Drops: 10
minDelta: 0.0000 us
maxDelta: 0.0000 us
meanDelta: 0.0000us
Standard deviation: 0.0000
[25-03-2024 10:46:15] Sensor statistics for: camera[0] camera.gmsl, camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216
[25-03-2024 10:46:15] Statistics for DW_SENSOR_STATE_DELTA_READ_AND_RETURN_FRAME_TIME
Events: 94
Errors: 0
CRC Errors: 0
Drops: 10
minDelta: 40992.0000 us
maxDelta: 77024.0000 us
meanDelta: 42166.0391us
Standard deviation: 5291.0229
Serializer statistics for sensor: camera:sample0
Statistics for DW_SERIALIZER_STAT_DISK_WRITE_TIME
minDelta: 0 us
maxDelta: 0 us
meanDelta: 0.000 us
standardDeviationDelta: 0.000
Statistics for DW_SERIALIZER_STAT_ENCODE_TIME
minDelta: 28288 us
maxDelta: 29440 us
meanDelta: 28534.043 us
standardDeviationDelta: 269.748
Statistics for DW_SERIALIZER_STAT_STAGE1_TIME
minDelta: 128 us
maxDelta: 672 us
meanDelta: 263.111 us
standardDeviationDelta: 162.734
Statistics for DW_SERIALIZER_STAT_STAGE2_TIME
minDelta: 128 us
maxDelta: 480 us
meanDelta: 277.333 us
standardDeviationDelta: 141.348
Statistics for DW_SERIALIZER_STAT_STAGE3_TIME
minDelta: 27712 us
maxDelta: 28384 us
meanDelta: 27993.602 us
standardDeviationDelta: 147.586
[25-03-2024 10:46:15] Serializer: drained data to
[25-03-2024 10:46:15] SIPLClient: Total number of frames with FrameAccumulator = 4
[25-03-2024 10:46:15] Buffers(Processed) should be released before calling DeInit() on sipl camera
[25-03-2024 10:46:16] SIPLDeviceBlockNotificationHandler: Queue timeout
[25-03-2024 10:46:16] SIPLClient: ImageQueue timeout
[25-03-2024 10:46:16] SIPLNotifier: Event notification queue timeout
[25-03-2024 10:46:16] Deinit master camera
[25-03-2024 10:46:16] SIPLNotifier: Event notification queue shutdown
[25-03-2024 10:46:16] CameraClient: Stopping client
[25-03-2024 10:46:16] Releasing camera master
[25-03-2024 10:46:17] SIPLDeviceBlockNotificationHandler: Queue timeout
[25-03-2024 10:46:17] Driveworks VisualizationSDK released
[25-03-2024 10:46:17] [25-03-2024 10:46:17] Releasing Driveworks SDK Context
[25-03-2024 10:46:17] DriveworksGL SDK released

The rig.json:
{
“rig”: {
“sensors”: [
{
“name”: “camera:sample0”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
0.0,
0.0,
0.0
],
“t”: [
1.8621,
-0.1939,
1.3165
]
},
“parameter”: “camera-name=F008A120RM0AV2,interface=csi-ef,CPHY-mode=1,link=0,output-format=processed,async-record=0,file-buffer-size=16777216”,
“properties”: {
“COMMENT”: “Nominals estimated for car27 / VIN=275155 (http://nvbugs/2869761/1)”,
“Model”: “ftheta”,
“bw-poly”: “0.000000000000000 5.35356812179089e-4 4.99266072928606e-10 4.27370422037554e-12 -6.68245573791717e-16”,
“cx”: “1927.764404”,
“cy”: “1096.686646”,
“height”: “2168”,
“width”: “3848”
},
“protocol”: “camera.gmsl”
}
],
“vehicle”: {
“valid”: false
}
},
“version”: 6
}

8

Dear @xiaolei.qin,
In my observation, to store 8.3 M camera with 30FPS, we need 8.3 * 30 * 12 * 1/8 = ~370 MBps bandwidth. I verified the the storage partition has BW around ~400MBps using dd command.
I could record 1 single camera data with out any issue using recorder tool. By using frame-mod=2(stores every alternate frame) parameter, I can use two cameras with out frame drops.

9

Hello @SivaRamaKrishnaNV ,
The main issue we are concerned about is that compressing an 8MP image with H264 takes about 10ms longer to test on driveos 6.0.8 than on driveos 6.0.4. Do I need to set any parameters to improve the performance?

10

You mean, The performance of h264 encoding is better in DRIVE OS 6.0.4? Can you share the sample code you have used to test the performance?

11

Hello @SivaRamaKrishnaNV ，
The code and rig file pls refer to the attach-file:
sample_camera.zip (10.7 KB)

The code base on the driveos 6.0.8/driveworks5.14 sample_camera,
mainly modification:
**1.**Add start time check point before calling dwSensorSerializer_serializeCameraFrameAsync //refer to line 883
**2.**Modified the type to user mode and make the callback(serializerParams.onData) works:
newParams += std::string(“,type=user”); //refer to line 390
serializerParams.onData = SensorSerializerOnDataFunc; //refer to line 404
**3.*Add end time point check to calclate the time spent.
SensorSerializerOnDataFunc(const uint8_t data, size_t size, void *userData) //refer to line 364

The key points:
1.Please help to check if there are any issues with our code. If there are any issues, please let us know.
2.If there are no issues with the code, can you provide your time report on H264 to compress 8MP images.

Another question: there was a flag named m_bUseNewCodecStack in the cpp, if I enable the m_bUseNewCodecStack = true may get the info below:
[26-03-2024 02:46:09] Support serialization to disk only. This is needed implicitly. Current serializer config param doesn’t have type value.
[26-03-2024 02:46:09] Driveworks exception thrown: DW_FILE_NOT_FOUND: FileStreamBuffered: could not open file
That means the mode m_bUseNewCodecStack = true will not support the callback serializerParams.onData?

12

Hello @SivaRamaKrishnaNV ,
Have you got any new updates about the codes?