Low-Latency CSI Camera Stream

Autonomous Machines Jetson & Embedded Systems Jetson Nano
#4

Have a check with the argus_camera below result is I got from Pi imx219.

nvidia@nvidia-desktop:~$ ./argus_camera --kpi --sensormode=3
Executing Argus Sample Application (argus_camera)
Argus Version: 0.97.3 (multi-process)
PerfTracker: app initial 556 ms
PerfTracker 1: app intialized to task start 663 ms
PerfTracker 1: task start to issue capture 73 ms
PerfTracker 1: first request 169 ms
PerfTracker 1: total launch time 1462 ms
PerfTracker 1: frameRate 75.65 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 24 ms average, min 18 max 29
PerfTracker 1: frameRate 61.78 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 17 ms average, min 15 max 21
PerfTracker 1: frameRate 60.64 frames per second at 1 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 17
PerfTracker: display frame rate 52.62 frames per second
PerfTracker 1: frameRate 60.69 frames per second at 1 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 19
PerfTracker 1: frameRate 60.52 frames per second at 1 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 17
PerfTracker: display frame rate 60.04 frames per second
PerfTracker 1: frameRate 60.42 frames per second at 2 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 17
PerfTracker 1: frameRate 60.34 frames per second at 2 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 17
PerfTracker: display frame rate 60.00 frames per second
PerfTracker 1: frameRate 60.32 frames per second at 3 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 16 ms average, min 15 max 17
1 Like
#5

With the same “argus_camera” program I get a very different result:

./argus_camera --kpi --sensormode=3
Executing Argus Sample Application (argus_camera)
Argus Version: 0.97.3 (multi-process)
PerfTracker: app initial 631 ms
PerfTracker 1: app intialized to task start 474 ms
PerfTracker 1: task start to issue capture 68 ms
PerfTracker 1: first request 1196 ms
PerfTracker 1: total launch time 2370 ms
PerfTracker 1: frameRate 2,01 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker: display frame rate 2,02 frames per second
PerfTracker 1: latency 51 ms average, min 51 max 53
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,02 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker 1: flush takes 2086 ms
PerfTracker 1: device close takes 8 ms
PerfTracker 1: total close takes 2095 ms

How can that be?

#6

Thanks for that answer. That is kind of the fallback solution, but right now I specifically try to evaluate the performance of libargus, respectively the ISPs from NVIDIA which are only accessible via libargus.

#7

Have a check with v4l2-ctl that the sensor cap can output 60fps correctlly.

nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=300
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.39 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.19 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.13 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.09 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
#8

Yes it can a leat get very close. See here:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=300
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 58.41 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 58.20 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 58.33 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 58.25 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 58.20 fps

That is very weird. Why is the argus_camera output then so different for us?

#9

Wow, that’s weird, which release for your problem.

#10

I have the following release:

NVIDIA Jetson Nano (Developer Kit Version)
L4T 32.4.3 [ JetPack 4.4 ]
Ubuntu 18.04.5 LTS
Kernel Version: 4.9.140-tegra
CUDA 10.2.89
CUDA Architecture: 5.3
Argus Version: 0.97.3
Multimedia API: 32.2

The argus_camera programm produced a segfault when running at first, so I patched the headers “CameraDevice.h” and “Settings.h” as described by the mod JerryChang here: https://forums.developer.nvidia.com/t/l4t-r32-4-3-breaks-argus-camera-and-causes-segmentation-fault-sigsegv/141432/8. With these tiny patches the argus_camera program works.

I did not modify the system otherwise.

Can the patches be the source of the slowness?

#11

I don’t think the source cause the problem.
Could you attached you binary here, I can verify it on my system.

#12

Of course. Here is the binary (packed with gzip as otherwise I can’t upload it):
argus_camera.gz (480.1 KB)

#15

Looks like your binary have problem. I just got 2 fps from this binary.

#16

How can that be? That should not be possible.
I compiled that binary with the newest Multimedia API from the NVIDIA Webpage and the workflow excactly as described in the README.txt file in the tegra_multimedia_api/argus folder.
Can you give me a binary without that problem for testing purposes?

#17

Have a check this binary it’s build from internal tree. I will try the release build later.

Just remove the .txt and chmod +x to run it.

argus_camera.txt (2.2 MB)

#18

I tried your binary and it leads to the same result as you can see below:

./argus_camera --kpi --sensormode=3
Executing Argus Sample Application (argus_camera)
Argus Version: 0.97.3 (multi-process)
PerfTracker: app initial 656 ms
PerfTracker 1: app intialized to task start 436 ms
PerfTracker 1: task start to issue capture 64 ms
PerfTracker 1: first request 1213 ms
PerfTracker 1: total launch time 2371 ms
PerfTracker 1: frameRate 2,01 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker: display frame rate 2,02 frames per second
PerfTracker 1: latency 51 ms average, min 51 max 53
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,02 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker: display frame rate 2,00 frames per second
PerfTracker 1: flush takes 1715 ms
PerfTracker 1: device close takes 3 ms
PerfTracker 1: total close takes 1719 ms

Is maybe some shared library on the system broken in my case, which is linked into all the libargus executeables and this leads to the slow FPS?
Also why are the FPS exactly 2,00? tTat seems very weird to me. It’s not just slow, it’s exactly 2.

My system is really a fresh and unmodified install from the latest SD card image provided by NVIDIA.

#19

My bad, attach the wrong file. Please try this one.

snchen@snchen-HP:~$ sha256sum /tmp/argus_camera
a1a0872c2801855ed837968fc0de5dde4be2c24c6f1f22f93a4196ad24b99652

argus_camera.txt (1.2 MB)

1 Like
#20

./argus_camera --kpi --sensormode=3
Executing Argus Sample Application (argus_camera)
Argus Version: 0.97.3 (multi-process)
PerfTracker: app initial 584 ms
PerfTracker 1: app intialized to task start 561 ms
PerfTracker 1: task start to issue capture 67 ms
PerfTracker 1: first request 152 ms
PerfTracker 1: total launch time 1365 ms
PerfTracker 1: frameRate 72,26 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 17 ms average, min 16 max 20
PerfTracker 1: frameRate 60,38 frames per second at 0 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 15 ms average, min 15 max 18
PerfTracker 1: frameRate 60,39 frames per second at 1 Seconds
PerfTracker 1: framedrop current request 0, total 0
PerfTracker 1: latency 15 ms average, min 15 max 16
PerfTracker: display frame rate 60,31 frames per second
PerfTracker 1: frameRate 60,29 frames per second at 1 Seconds
PerfTracker 1: flush takes 87 ms
PerfTracker 1: device close takes 12 ms
PerfTracker 1: total close takes 100 ms

It works. But why? What is different?
Also your executeable is much smaller in file size. Mine is 2.2MB, yours us 1.2MB

#21

There’s no problem for me to build from the public release source too.
The size is about 2.2MB too.
Attached my source here for reference.argus.tar.gz (2.0 MB)

#22

Using the sources you provided I cannot compile with the commands as written in README.txt. The make command fails with:

In file included from /home/nico/testmod/samples/utils/gtk/Window.h:32:0,
from /home/nico/testmod/samples/utils/Window.h:34,
from /home/nico/testmod/apps/camera/renderer/Composer.h:36,
from /home/nico/testmod/apps/camera/renderer/Composer.cpp:40:
/home/nico/testmod/samples/utils/WindowBase.h:32:10: fatal error: Argus/Argus.h: No such file or directory
#include <Argus/Argus.h>

#23

I now managed to compile. I realized I have to copy it into the MMAPI folder to be able to compile.

And a wonder happened! The resulting executable runs smoothly with 60FPS!!!

#24

I also found out what the problem is because I made a diff of your uploaded source code to my source code from the NVIDIA download center.

The download center provides an outdated version of the multimedia API if one activates the Jetson Nano filter on the left. See here:

Then the newest version that is shown is

32.2 from 2019/07/18

if one leaves away the Filter, the newest Multimedia API version that has a dedicated Nano download link is:

32.2.3 from 2019/11/19

(but in this version the argus_camera app has the Segmentation Fault again as described in L4T R32.4.3 Breaks argus_camera and causes segmentation fault (SIGSEGV))

I think that is clearly a bug on the NVIDIA homepage and NVIDIA should fix this.

Could you confirm what is the newest Multimedia API package that works for the Jetson Nano without segfaulting and with high FPS?

#25

Please download the Multimedia API source from the sdkmanager in future for sync with the JetPack version.