Drive AGX Thor Exposure Start Timing Control

Autonomous Vehicles DRIVE AGX Thor DRIVE AGX Thor General
1

DRIVE OS Version: Provide DRIVE OS version. Example: 7.0.3

Issue Description:
I am using Drive AGX Thor (DriveOS 7.0.3) with an IMX728 camera and acquiring camera frames through the DriveWorks API.
What I would like to achieve is to control the camera shutter timing (exposure start timing) so that exposure starts at a specific time.
For example, I would like the camera exposure to start at hh:mm:ss.000 (system time) and then continue at 30 FPS.

I have the following questions:

1.
Is my understanding correct that this type of exposure start timing control cannot be achieved directly through the DriveWorks API, and that it should instead be implemented using FSYNC?

2.
If the answer to Question 1 is yes, can the following approach be used to achieve the desired behavior?

  • Obtain the current TSC value using NvpGetTimeMark().
  • Establish the relationship between the current system time and the TSC value.
  • Calculate the TSC corresponding to the desired exposure start time.
  • Execute the following command:
    fsync_debug startgroup 0 <calculated_TSC>

Will this allow the IMX728 exposure to start at the specified time by starting FSYNC at the calculated TSC value?
If this approach is incorrect, could you please advise the recommended method for synchronizing the exposure start timing to a specific timestamp in a Drive AGX Thor + IMX728 + DriveWorks environment?

Thank you for your support.

4

Dear @shoji.watanabe.01 ,
Could you please refer to below topics and see if it helps to clarify?

5

Thank you for the information. It is helpful, but it is slightly different from what I would like to confirm.
Change the question

I would like to confirm the behavior of the following command:

fsync_debug startgroup <GROUP_IDX> <TSC_START_VALUE_TICKS>

・Is it correct to understand that <TSC_START_VALUE_TICKS> represents the TSC timestamp at which the FSYNC signal is generated?
・Also, is it correct to interpret that:
FSYNC = first row exposure start of the CMOS sensor

I performed the following verification.

After starting FSYNC with fsync_debug, I checked the SOF timestamps using nvsipl_camera.

fsync_debug startgroup 0 93,958,663,983

./nvsipl_camera --platform-config “R0SIM728S3RU2120NB2_5Des_CPHY_x2”
–link-enable-masks “0x0000 0x00 0x0000 0x0000 0x0001”
–showfps -1 -2 -M

The TSC SOF timestamps for three consecutive frames were:

TSC SOF: 98,903,098,400
TSC SOF: 98,936,432,125
TSC SOF: 98,969,765,774

The exposure time was configured to 10 ms.
According to the explanation provided in the following thread:

first row exposure start = sofTimestampUs - exposureDurationUs[0]

the estimated first row exposure start timestamps are:

98,903,098,400 - 10,000,000 = 98,893,098,400
98,936,432,125 - 10,000,000 = 98,926,432,125
98,969,765,774 - 10,000,000 = 98,959,765,774

These values are close to:

<TSC_START_VALUE_TICKS> + (n × 33,333,333)

※The difference is approximately 1.101 ms.

Based on these results, my understanding is:
・<TSC_START_VALUE_TICKS> determines the timing of FSYNC generation.
・FSYNC affects the first row exposure start timing.

Furthermore, I assume that the observed 1.101 ms difference is caused by latency in one or both of the following parts:

<TSC_START_VALUE_TICKS>
↓ x.xxx[ms]
FSYNC generation
↓ x.xxx[ms]
First row exposure start

In other words, either:
there is a delay between <TSC_START_VALUE_TICKS> and the actual FSYNC generation,
there is a delay between FSYNC generation and the first row exposure start,
or both,

Is this understanding correct?

6

@SivaRamaKrishnaNV is there any update or do you need other information?

8

Could you please provide any update for this topic?

9

Thank you for the follow-up.

At this point, I do not have any new information to provide. I am still waiting for a response to the questions I raised in this topic.

If there is any information missing from my previous description or if additional details are required to investigate this issue, please let me know. I would be happy to provide further clarification.

10

Dear @shoji.watanabe.01 ,

Just checking, Do you have any dependency on DOS 7.0.3 or can be upgraded to DOS 7.2.5. We are working on DOS 7.2.5 patch which allows the DW to start capturing with fsync-1-offset-ns rig parameter which is equivalent equivalent to cam_fsync_program_abs_start_value (Camera Frame Synchronization — NVIDIA DriveOS 7.2.5 Linux SDK Early Access Developer Guide) . Does it help your case?

Could you explain this how you got this number? that means you notice delay of 1.101 ms between first frame SOF timestamp and TSC_start_value_ticks timestamp?

11

Thank you for the information.

Just checking, Do you have any dependency on DOS 7.0.3 or can be upgraded to DOS 7.2.5. We are working on DOS 7.2.5 patch which allows the DW to start capturing with fsync-1-offset-ns rig parameter which is equivalent equivalent to cam_fsync_program_abs_start_value ( Camera Frame Synchronization — NVIDIA DriveOS 7.2.5 Linux SDK Early Access Developer Guide ) . Does it help your case?

If this issue cannot be resolved on DriveOS 7.0.3, we are willing to consider upgrading to DriveOS 7.2.5.

However, my understanding is that cam_fsync_program_abs_start_value is already available in DriveOS 7.0.3. Is the patch for DriveOS 7.2.5 intended to address a known issue or limitation with the existing implementation?

Could you explain this how you got this number? that means you notice delay of 1.101 ms between first frame SOF timestamp and TSC_start_value_ticks timestamp?

<TSC_START_VALUE_TICKS> in fsync_debug startgroup is assumed to be the exposure start timing (first row exposure start) of the first frame.

For example, when we set:

TSC_START_VALUE_TICKS = 93,958,663,983

Frame period = 33,333,333 ns (30 fps)

we expect the first row exposure start timing to be:
we assume that TSC_START_VALUE_TICKS corresponds to the first row exposure start timing

Frame1: 93,958,663,983
Frame2: 93,991,997,316
Frame3: 94,025,330,649
Frame4: 94,058,663,982

Therefore, the lower digits of the expected exposure start timestamps repeat with the following pattern:

xx,x58,663,983
xx,x91,997,316
xx,x25,330,649

Based on the frame information, the exposure start timing was calculated as follows.
first row exposure start = sofTimestampUs - exposureDurationUs[0]

and obtained:

98,893,098,400
98,926,432,125
98,959,765,774

Comparing these values with the expected exposure start timing:

98,893,098,400 - xx,x91,997,316 = 1,101,084 ns
98,926,432,125 - xx,x25,330,649 = 1,101,476 ns
98,959,765,774 - xx,x58,663,983 = 1,101,791 ns

The difference is consistently around 1.1 ms.

For this reason, we concluded that the actual first row exposure start occurs approximately 1.1 ms later than the timing expected from TSC_START_VALUE_TICKS.

13

This gap can be attributed additional internal delays between Fsync signal generator start to SOF.

The patch is intended to make DW to use flag in rig file to enable fsync instead of cam_fsync_program_abs_start_value(). Note that camera FSYNC feature is not POR in DW.

14

This gap can be attributed additional internal delays between Fsync signal generator start to SOF.

The patch is intended to make DW to use flag in rig file to enable fsync instead of cam_fsync_program_abs_start_value(). Note that camera FSYNC feature is not POR in DW.

16

Thank you for your reply.

I close this ticket