Jetson sensor external trigger

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hello,

What is necessary to support external sensor trigger with custom sensor driver on Nano A02/B01 (later Xavier, TX2) / 32.3.1.

Currently it seems that an external trigger is processed OK only when it comes at a moment when the capture logic is in a given state. In the rest of the time it is ignored.

Greetings

3

What’s external sensor trigger??

4

Our sensors have two modes: free run and trigger mode. When we set the sensor in trigger mode and start streaming, the sensor waits for external signal (raising edge from 0 to 3.3V on a wire from the connecting cable)
and then sends frame data to Jetson. The triggers come in unknown time intervals, so Jetson should wait infinite time for coming frame(s).

At the moment vi2_fops.c supports free-run (immediate frame data) and performs retry/recover operations when the frame data has not come (timeout 200 ms). How can we change this behavior to support infinite waiting for frame data ?

5

Sorry, to tell current driver design not support trigger mode.

6

Here:
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-321/Tegra%20Linux%20Driver%20Package%20Development%20Guide/jetson_xavier_camera_soft_archi.html#wwpID0E0OC0HA
we found the following info:

Infinite Timeout Support

This use case is different from typical camera use cases, where the camera
sensor streams frames continuously and finite timeouts for the camera driver and associated hardware waiting for camera frames.

In this use case a camera sensor is triggered to generate a specified number of frames, after which it stops streaming indefinitely. Whenever it resumes streaming, the camera driver must resume capturing without timeout issues. The camera driver and hardware must always be ready to capture frames coming from the CSI sensor. Since the camera driver does not know when streaming will start again, it must wait indefinitely for incoming frames.

To support this use case, the camera sensor hardware module must support suspending and resuming streaming. To enable this feature, run the camera server (i.e. the nvargus-daemon service) by setting an environment variable:

sudo service nvargus-daemon stop
sudo enableCamInfiniteTimeout=1 nvargus-daemon

May be there are other means to get frames in trigger mode ?

7

This mode was for debug purpose. However you can try on your use case.

8

Thank you ShaneCCC,

Well, just to be sure, I will summarize:

  1. We have created custom V4L driver for MIPI sensor. With current L4T kernel we can’t use our V4L driver in trigger mode with infinite waiting for frame data ?
  2. May be we can use libargus for this purpose. If so, can you please give us example how to capture frames in trigger mode ?
9
  1. Yes, current VI mode design as continuous capture mode. However the VI driver is public you can take time to modify it as your request.
  2. You can check the multimedia API same “oneshot” to check how to issue a capture and may need develop side band to communicate to trigger the sensor.
10

We tested the argus service nvargus-daemon on Nano A02 32.3.1. We used fresh installed OS, flashed on SD card from the Nvidia supplied image without any modifications. The connected sensor s IMX219 - RPI V2 camera, supported by default.

However the enableCamInfiniteTimeout seems to fail:

sudo service nvargus-daemon stop
sudo enableCamInfiniteTimeout=1 nvargus-daemon &
[1] 7677
=== NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=955, height=616' ! nvjpegenc ! filesink location=imx219_2.jpg

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
=== gst-launch-1.0[7684]: Connection established (7FAEAC11D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== gst-launch-1.0[7684]: CameraProvider initialized (0x7fa8837850)SCF: Error BadParameter:  (propagating from src/services/capture/NvViCsiHw.cpp, function openViCsi(), line 117)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function open(), line 307)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function openSource(), line 355)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function openSource(), line 478)
SCF: Error BadParameter:  (propagating from src/api/Session.cpp, function initialize(), line 262)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function createSession(), line 569)
(Argus) Error BadParameter:  (propagating from src/api/CaptureSessionImpl.cpp, function initialize(), line 120)
(Argus) Error BadParameter:  (propagating from src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 250)
(NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/argus/src/api/CaptureSessionImpl.cpp:197) (in Mutex.cpp, function lock(), line 79)
(Argus) Error InvalidState: Element not found (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function remove(), line 172)
(NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:543 Failed to create CaptureSession
Got EOS from element "pipeline0".
Execution ended after 0:00:00.578161505
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
=== gst-launch-1.0[7684]: CameraProvider destroyed (0x7fa8837850)=== gst-launch-1.0[7684]: Connection closed (7FAEAC11D0)=== gst-launch-1.0[7684]: Connection cleaned up (7FAEAC11D0)

The same GStreamer pipeline works OK when nvargus-daemon is restarted without enableCamInfiniteTimeout=1 option.

11

IMX219 should be 3264x2464 ?

12

OK,
With corrected params the effect is the same.
BTW: Actually Jetson 32.3.1 ignores incorrect params and uses the nearest correct values.

13

Could you check if nvargus_camera can run for enableCamInfiniteTimeout=1

14

What exactly do you mean by “nvargus_camera”? We found this only – is this what you mean?
GitHub - NVIDIA-AI-IOT/argus_camera: Simple Python / C++ interface to CSI camera connected to NVIDIA Jetson.
It fails to compile – probably it was developed for an old API version.

However, we tested with some of the API samples. Here are the results:

Testing with argus_oneshot:

sudo enableCamInfiniteTimeout=1 nvargus-daemon
=== NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...===
argus_oneshot[8626]: Connection established (7F8C7BE1D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== argus_oneshot[8626]: CameraProvider initialized (0x7f84828e50)SCF: Error BadParameter: (propagating from src/services/capture/NvViCsiHw.cpp, function openViCsi(), line 117)
SCF: Error BadParameter: (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function open(), line 307)
SCF: Error BadParameter: (propagating from src/services/capture/CaptureServiceDevice.cpp, function openSource(), line 355)
SCF: Error BadParameter: (propagating from src/services/capture/CaptureService.cpp, function openSource(), line 478)
SCF: Error BadParameter: (propagating from src/api/Session.cpp, function initialize(), line 262)
SCF: Error BadParameter: (propagating from src/api/CameraDriver.cpp, function createSession(), line 569)
(Argus) Error BadParameter: (propagating from src/api/CaptureSessionImpl.cpp, function initialize(), line 120)
(Argus) Error BadParameter: (propagating from src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 250)
(NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/argus/src/api/CaptureSessionImpl.cpp:197) (in Mutex.cpp, function lock(), line 79)
(Argus) Error InvalidState: Element not found (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function remove(), line 172)
(NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
=== argus_oneshot[8626]: CameraProvider destroyed (0x7f84828e50)=== argus_oneshot[8626]: Connection closed (7F8C7BE1D0)=== argus_oneshot[8626]: Connection cleaned up (7F8C7BE1D0)

When the daemon is started normally (without the enableCamInfiniteTimeout=1 switch) this example works fine and produces a jpeg image.

The same result when testing with 10_camera_recording.

We found similar demo here:

Maybe this option is supported by Xavier but not Nano.

15

Hello Nvidia support,

Please answer to our question.
We really need to know does argus on Jetson Nano support external sensor trigger by the enableCamInfiniteTimeout option.
Our clients are very unhappy when unable to use external sensor triggers.

This problem seems to be a bug. If so, please say:

  1. Jetson Nano does not support enableCamInfiniteTimeout.
    If not:
  2. Please send us instructions how to use it.
16

@plamenk
Just confirm internally Nano/TX1 not support enableCamInfiniteTimeout but TX2 and after chip.