sensor manager & sensor serializer & timestamp

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Hi, I’m trying to implement sensor(camera & can) data recording with sensor manager & sensor serializer. And I have two question about that.

  1. I succeed recording CAN and Camera data. During recording, data is loaded sequentially and saved well. But after recorded and trying to replay, timestamp of camera data is start to 0, but CAN data is start to recorded timestamp.

for example,

TimeStamp Sensor Type

0 CAMERA
100000 CAMERA
200000 CAMERA
300000 CAMERA
400000 CAMERA
500000 CAMERA
600000 CAMERA
700000 CAMERA
…
1499163130429887 CAN
1499163130799851 CAN
1499163131173080 CAN
1499163131497844 CAN
1499163131868557 CAN
1499163132238348 CAN

like that. How to match the timestamp??

  1. I implemented data recording with dwSensorManager and dwSensorSerializer. So I checked that camera data is recorded with sensor manager and serializer.
    But in case of CAN data, RawData is empty when i received with sensormanager.

for example,

dwSensorManager_acquireNextEvent(&sensor_event, 0, dw_sensor_mng);
if(sensor_event->type == DW_SENSOR_CAN)
{
sensor_event->rawDataSize is zero and
sensor_event->rawData is empty.
}

so I implemented like below,

dwSensorManager_acquireNextEvent(&sensor_event, 0, dw_sensor_mng);
if(sensor_event->type == DW_SENSOR_CAN)
{
dwSensorManager_getSensorHandle(&can_handle, event->sensorIndex, dw_sensor_mng);
dwSensor_readRawData(&data, &size, 100000, can_handle);
recording.RecordCanData(event->sensorTypeIndex, data, size); //this is recording with serializer

dwSensor_returnRawData(data, can_handle);

}

is it correct that rawDataSize is zero, so user have to get SensorHandle and read RawData??

2

Dear jw31,
Did you check using our recording tool. You can record camera, can data with that? It is not serving the purpose? Do you see any gaps in the tool?

3

Today I checked using nvidia recording tool. But same problems occur.
(timestamp of CAN and Camera are different)

Here is my rig configuration

{
“rig”: {
“properties”: {
“layout”: “Hyperion 7.0”
},
“sensors”: [
{
“correction_rig_T”: [
0.0,
0.0,
0.0
],
“correction_sensor_R_FLU”: {
“roll-pitch-yaw”: [
0.03438815474510193,
-3.7623074054718018,
0.16219370067119598
]
},
“name”: “camera:front:center:60fov”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
-5.798638449050486e-05,
1.1458930969238281,
-0.5729333758354187
],
“t”: [
1.7489999532699585,
-0.10000000149011612,
1.4700000286102295
]
},
“parameter”: “camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=4,siblingIndex=0,slave=0,format=h264,fifo-size=4,isp-mode=yuv420-uint8,bitrate=8000000,framerate=10,type=disk,file=video1.h264”,
“properties”: {
“Model”: “ftheta”,
“bw-poly”: “0.0 0.000545421498827636 -1.6216719633103e-10 -4.64720492990289e-12 2.85224527762934e-16”,
“cx”: “960”,
“cy”: “604”,
“height”: “1208”,
“width”: “1920”
},
“protocol”: “camera.gmsl”
},
{
“correction_rig_T”: [
0.0,
0.0,
0.0
],
“correction_sensor_R_FLU”: {
“roll-pitch-yaw”: [
0.03438815474510193,
-3.7623074054718018,
0.16219370067119598
]
},
“name”: “camera:front:left:60fov”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
-5.798638449050486e-05,
1.1458930969238281,
-0.5729333758354187
],
“t”: [
1.7489999532699585,
-0.10000000149011612,
1.4700000286102295
]
},
“parameter”: “camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=4,siblingIndex=1,slave=0,format=h264,fifo-size=4,isp-mode=yuv420-uint8,bitrate=8000000,framerate=10,type=disk,file=video2.h264”,
“properties”: {
“Model”: “ftheta”,
“bw-poly”: “0.0 0.000545421498827636 -1.6216719633103e-10 -4.64720492990289e-12 2.85224527762934e-16”,
“cx”: “960”,
“cy”: “604”,
“height”: “1208”,
“width”: “1920”
},
“protocol”: “camera.gmsl”
},
{
“correction_rig_T”: [
0.0,
0.0,
0.0
],
“correction_sensor_R_FLU”: {
“roll-pitch-yaw”: [
0.03438815474510193,
-3.7623074054718018,
0.16219370067119598
]
},
“name”: “camera:front:right:60fov”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
-5.798638449050486e-05,
1.1458930969238281,
-0.5729333758354187
],
“t”: [
1.7489999532699585,
-0.10000000149011612,
1.4700000286102295
]
},
“parameter”: “camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=4,siblingIndex=2,slave=0,format=h264,fifo-size=4,isp-mode=yuv420-uint8,bitrate=8000000,framerate=10,type=disk,file=video3.h264”,
“properties”: {
“Model”: “ftheta”,
“bw-poly”: “0.0 0.000545421498827636 -1.6216719633103e-10 -4.64720492990289e-12 2.85224527762934e-16”,
“cx”: “960”,
“cy”: “604”,
“height”: “1208”,
“width”: “1920”
},
“protocol”: “camera.gmsl”
},
{
“correction_rig_T”: [
0.0,
0.0,
0.0
],
“correction_sensor_R_FLU”: {
“roll-pitch-yaw”: [
0.03438815474510193,
-3.7623074054718018,
0.16219370067119598
]
},
“name”: “camera:front:rear:60fov”,
“nominalSensor2Rig_FLU”: {
“roll-pitch-yaw”: [
-5.798638449050486e-05,
1.1458930969238281,
-0.5729333758354187
],
“t”: [
1.7489999532699585,
-0.10000000149011612,
1.4700000286102295
]
},
“parameter”: “camera-type=ar0231-rccb-bae-sf3324,camera-group=a,camera-count=4,siblingIndex=3,slave=0,format=h264,fifo-size=4,isp-mode=yuv420-uint8,bitrate=8000000,framerate=10,type=disk,file=video4.h264”,
“properties”: {
“Model”: “ftheta”,
“bw-poly”: “0.0 0.000545421498827636 -1.6216719633103e-10 -4.64720492990289e-12 2.85224527762934e-16”,
“cx”: “960”,
“cy”: “604”,
“height”: “1208”,
“width”: “1920”
},
“protocol”: “camera.gmsl”
},
{
“name”: “can:vehicle:generic”,
“parameter”: “device=can1,type=disk,file=can1.bin,serialize=true”,
“protocol”: “can.socket”,
“properties”: null,
“nominalSensor2Rig”: { “quaternion”: [ 0.0, 0.0, 0.0, 1.0 ], “t”: [ 0.0, 0.0, 0.0 ] },
“sensor2Rig”: { “quaternion”: [ 0.0, 0.0, 0.0, 1.0 ], “t”: [ 0.0, 0.0, 0.0 ] }
}
],
“vehicle”: {
“valid”: true,
“value”: {
“axlebaseFront”: 1.582419991493225,
“axlebaseRear”: 1.5748000144958496,
“bumperFront”: 0.9120000004768372,
“bumperRear”: 1.1090999841690063,
“centerOfMassToRearAxle”: 1.5644999742507935,
“driveByWireTimeConstant”: 0.25999999046325684,
“driveByWireTimeDelay”: 0.10999999940395355,
“frontCorneringStiffness”: 30654.0,
“height”: 1.4731999635696411,
“inertia”: 1780.800048828125,
“length”: 4.871719837188721,
“mass”: 1779.4000244140625,
“rearCorneringStiffness”: 36407.0,
“steeringCoefficient”: 14.800000190734863,
“wheelDiameter”: 0.672702968120575,
“wheelbase”: 2.8498799800872803,
“width”: 1.8516600131988525,
“widthWithMirrors”: 0.0,
“driveByWireTimeDelay”: 0.11,
“aerodynamicDragCoeff”: 0.7,
“frontalArea”: 1.0,
“centerOfMassHeight”: 1.5,
“aeroHeight”: 1.5,
“rollingResistanceCoeff”: 0.015,
“maxEnginePower”: 134e3,
“throttleActuatorTimeConstant”: 1.0,
“brakeActuatorTimeConstant”: 0.5,
“centerOfMassToFrontAxle”: 1.286,
“throttleStateInput”: [0, 0.07, 0.14, 0.21, 0.29, 0.36, 0.43, 0.50, 0.57, 0.64, 0.71, 0.79, 0.86, 0.93, 1.00],
“throttleForceOutput”: [0, 312, 623, 1086, 2646, 4272, 5370, 6052, 6363, 6556, 6730, 6905, 7092, 7283, 7473],
“brakeStateInput”: [0, 0.07, 0.14, 0.21, 0.29, 0.36, 0.43, 0.50, 0.57, 0.64, 0.71, 0.79, 0.86, 0.93, 1.00],
“brakeForceOutput”: [0, 714, 1423, 2143, 2857, 3571, 4286, 5000, 5714, 6429, 7143, 7857, 8571, 9286, 10000]
}
}
},
“version”: 2
}

4

Dear jw31,
Could you confirm if you are using DRIVE SW 9.0? We are checking this issue with internal team and update you.

5

It’s DRIVE SW 8.0.

6

If possible, Could you please check this symptom on DRIVE SW 9.0 and let us know

7

You mean that we have to re-flash with DRIVE SW9.0 to just test this symptom?

I think it is risky to us.

8

Dear jw31,
This issue is fixed in DRIVE SW 9.0(DW version 2.0) Could you please check.

9

Thank you. I will consider it.

And Issue2 is also fixed on DRIVE SW9.0?

Issue 2:
2. I implemented data recording with dwSensorManager and dwSensorSerializer. So I checked that camera data is recorded with sensor manager and serializer.
But in case of CAN data, RawData is empty when i received with sensormanager.

10

Dear jw31,
Did you check the issue on DRIVE SW 9.0? If you have not tested yet, could you please provide a sample snippet which reproduces the issue. I will check and get back to you.

11

Dear jw31,
Could you give any update regarding this issue.