Time shift CSI stream?

Hi there!

I’m new to the capabilities of the TX2 video engine but I was wondering about something …

I have an IMX185 camera which is streaming video into the TX2 with a fixed speed of 25fps.
The IMX185 does not have any hardware trigger or synchronization features by itself.

Imagine you’re filming a LED being blinked at the same speed (25Hz) then depending on when in time the camera stream would be turned on the LED would either be constant off or constant on (as it is running at the same speed as the camera)

The LED is being blinked by an external circuit and not the Jetson, however, I have the LED tied into a GPIO input on the Jetson so my hope was that somehow I could synchronize the video stream to its rising edge using interrupts or something.

One way I could think off (quite advanced) would be to start a timer off the Start Of Frame interrupt from the MIPI feed (if that is even possible?) and then time the difference between the GPIO and the SOF signal - then I could either increase or decrease the camera frame rate (not sure if this can be done in real time?) to make them come as close to each other as possible.

Something like this:


Any suggestions or pointers ?

hello LasseRoedtnes,

you may have implementation in kernel drivers to control GPIO, that’ll close to the SOF.
if you look into sensor kernel drivers, there’s a capture thread to issue sensor capture frames.
for example,
$L4T_Sources/r32.4.2/Linux_for_Tegra/source/public/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi4_fops.c

static int tegra_channel_kthread_capture_start(void *data)
{
...
                err = tegra_channel_capture_frame(chan, buf);

it’ll using sync-point for waiting sensor start-of-frame, and programming the capture buffers.
for example,

static bool vi_notify_wait(struct tegra_channel *chan,
                struct tegra_channel_buffer *buf,
                struct timespec *ts)
{
...
                err = nvhost_syncpt_wait_timeout_ext(chan->vi->ndev,
                                chan->syncpt[i][SOF_SYNCPT_IDX], thresh[i],
                                chan->timeout, NULL, NULL);

Hi @JerryChang

Allow me to ask a few questions for clarification, please bear with me as I’m not expert on V4L etc.

The functions you mentioned provides a time stamp of when the SOF occured if I understand it correctly which is the information I need to compare against when my GPIO triggered to determine how far they are apart in time.

My next question would then be - is it possible to (on the fly with the stream running) change the frame rate or somehow pause the SOF for a few miliseconds or microseconds corresponding to the delay that i have figured out there is between my GPIO rising edge and the SOF?

Yours truly
Lasse

hello LasseRoedtnes,

you cannot pause SOF since it’s signal coming from sensor hardware, and pausing it will trigger VI driver timeouts.
you may also refer to multi-thread approach in VI-4 driver, which gather SOF, and EOF interrupts.

#SOF
static bool vi_notify_wait(struct tegra_channel *chan,
                struct tegra_channel_buffer *buf,
                struct timespec *ts)
...
        for (i = 0; i < chan->valid_ports; i++) {
                err = nvhost_syncpt_wait_timeout_ext(chan->vi->ndev,
                                chan->syncpt[i][SOF_SYNCPT_IDX], thresh[i],
                                chan->timeout, NULL, NULL);
#EOF
static void tegra_channel_release_frame(struct tegra_channel *chan,
                                struct tegra_channel_buffer *buf)
{
...
        for (i = 0; i < chan->valid_ports; i++) {
                err = nvhost_syncpt_wait_timeout_ext(chan->vi->ndev,
                                chan->syncpt[i][FE_SYNCPT_IDX],
                                buf->thresh[i],
                                chan->timeout, NULL, NULL);

due to the traveling time between SOF and EOF would be 1/FPS seconds,
you may compare these two interrupt with your GPIO raise edge and falling edge for the time difference.
thanks