Libv4l2_nvargus.so source code

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hi, I have made a “dummy driver” based on imx185 source code. The idea is to have a camera driver without i2c communication, so the driver probe is always ok. Then under user space I am able to send any i2c commands to the different cameras we have. This is only for testing and developing new camera modules.
So, I did that concept for the first version of the jetson nano module (only one CSI connector) and now I’m trying to port this from the jetpack 4.3 to the jetpack 4.5 and jetson nano with two CSI connectors.
The first version was all OK, but now I have a segmentation fault when I try to make a call to “v4l2_open” function.

If I run the test under valgrind I cann see this:

medios@mediosjn:~/v4l2libs/libv4l2$ valgrind ~/camtest
==16890== Memcheck, a memory error detector
==16890== Copyright (C) 2002-2017, and GNU GPL’d, by Julian Seward et al.
==16890== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==16890== Command: /home/medios/camtest
==16890==
Opening in BLOCKING MODE
==16890== Invalid read of size 8
==16890== at 0x4FF7090: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF7F9B: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF68F7: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF66B7: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4891CCB: v4l2_plugin_init (in /usr/lib/aarch64-linux-gnu/tegra/libnvv4l2.so)
==16890== Address 0x0 is not stack’d, malloc’d or (recently) free’d
==16890==
==16890==
==16890== Process terminating with default action of signal 11 (SIGSEGV)
==16890== Access not within mapped region at address 0x0
==16890== at 0x4FF7090: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF7F9B: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF68F7: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4FF66B7: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so)
==16890== by 0x4891CCB: v4l2_plugin_init (in /usr/lib/aarch64-linux-gnu/tegra/libnvv4l2.so)
==16890== If you believe this happened as a result of a stack
==16890== overflow in your program’s main thread (unlikely but
==16890== possible), you can try to increase the size of the
==16890== main thread stack using the --main-stacksize= flag.
==16890== The main thread stack size used in this run was 8388608.
==16890==
==16890== HEAP SUMMARY:
==16890== in use at exit: 959,896 bytes in 1,201 blocks
==16890== total heap usage: 1,641 allocs, 440 frees, 1,398,238 bytes allocated
==16890==
==16890== LEAK SUMMARY:
==16890== definitely lost: 58 bytes in 1 blocks
==16890== indirectly lost: 0 bytes in 0 blocks
==16890== possibly lost: 312,223 bytes in 37 blocks
==16890== still reachable: 647,615 bytes in 1,163 blocks
==16890== of which reachable via heuristic:
==16890== newarray : 4,736 bytes in 28 blocks
==16890== multipleinheritance: 1,184 bytes in 1 blocks
==16890== suppressed: 0 bytes in 0 blocks
==16890== Rerun with --leak-check=full to see details of leaked memory
==16890==
==16890== For counts of detected and suppressed errors, rerun with: -v
==16890== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

But I can not debug inside the init function of libv4l2_nvargus.so file because I’m not able to find the source code.
Maybe it is not released?
I’m not sure how I can continue to debug. Some Idea?

Thanks.

3

hello jaescobar.sanchez,

as you can see from r32.5 Driver Details, we don’t public libv4l2_nvargus source code.

4

could you please refer to below for an example of using dual camera on Jetson Nano,
thanks

$L4T_Sources/r32.5/Linux_for_Tegra/source/public/hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-porg-camera-rbpcv2-dual-imx219.dtsi

5

Hello,

I’ve made the driver and it works. As I said with the jetpack 4.3 I have no problems. With 4.5 I it works through opencv: cv2.VideoCapture(0), and it also works with gstreamer. The problem is that as I have different sensors, one of them is RAW12, so I need to go through v4l2 directly. If I build the example “camera_unit_sample” it breaks on v4l2_open function. I am testing with a sample build that only is calling to the open function and it breaks with the segmentation fault problem. I have plugged a USB webcam, then I see a new /dev/video2 device, and I have the same problem with the webcam. This is why I suspect that maybe the problem is not in my driver, but of course it could be.
It always works also with v4l2-ctl. These are the working calls:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=1080,pixelformat=RG12 --set-ctrl=sensor_mode=1 --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=pic.raw

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=712,pixelformat=UYVY --set-ctrl=sensor_mode=0 --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=pic.raw

gst-launch-1.0 -vvv v4l2src ! “video/x-raw, width=(int)1920, height=(int)712, format=(string)UYVY, framerate=(fraction)60/1” ! xvimagesink

And the example from:

https://linuxtv.org/downloads/v4l-dvb-apis/userspace-api/v4l/v4l2grab.c.html

breaks in the same v4l2_open function.

6

hello jaescobar.sanchez,

it looks you’re using v4l2 standard control to access video0 with different pixel format types.
is your sensor output different pixel format for different sensor modes? had you also check the raw files for confirmation?
thanks

7

Hello JerryChang,

Yes, different modes really means different sensors and different pixel format types. I have a board with a ds90ub954 deserializer always plugged into the jetson, and then different camera modules with different sensors that I can connect to this board through an FPDLinkIII connection.
I can test the driver with only one mode and one camera to see if it woks, but as I said before, this concept works well with the jetpack4.3. I am only upgrading to 4.5 because it would be nice to have the latest features. Anyway I will try.

Thank you.

8

hello jaescobar.sanchez,

please also check the sensor modes reported by v4l2 standard control,
for example, $ v4l2-ctl -d /dev/video0 --list-formats-ext
thanks

9

This is the output:

medios@mediosjn:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT

    Index       : 0
    Type        : Video Capture
    Pixel Format: 'RG12'
    Name        : 12-bit Bayer RGRG/GBGB
            Size: Discrete 1920x712
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x1080
                    Interval: Discrete 0.017s (60.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
            Size: Discrete 1920x712
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x1080
                    Interval: Discrete 0.017s (60.000 fps)

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'NV16'
    Name        : Y/CbCr 4:2:2
            Size: Discrete 1920x712
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x1080
                    Interval: Discrete 0.017s (60.000 fps)

    Index       : 3
    Type        : Video Capture
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
            Size: Discrete 1920x712
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x1080
                    Interval: Discrete 0.017s (60.000 fps)
10

hello jaescobar.sanchez,

so this is not a correct pipeline,

you must specify correct pixel format and resolution to the v4l2 command line for verification,
thanks

11

Hello JerryChang,

I’m sorry, maybe I’m not explaining well, but this command works well. And RG12 and 1280x1080 are the right pixel format and resolution. If I execute the command I have the right raw capture from the sensor (in all the modes).
The problem is when inside a program I call the function v4l2_open(“/dev/video0”).
The open is before any setting, and I’m getting a segmentation fault.
Maybe I’m missing something.

Thank you.

12

Hello,

If I build again the example from:

https://linuxtv.org/downloads/v4l-dvb-apis/userspace-api/v4l/v4l2grab.c.html

It works only modifying the line:

fd = v4l2_open(dev_name, O_RDWR | O_NONBLOCK, 0);

to

fd = open(dev_name, O_RDWR | O_NONBLOCK, 0);

One difference with the first call is that it outputs the message “Opening in BLOCKING MODE” before the segmentation fault, so it seems the flag O_NONBLOCK has no effect.

I don’t know if this can be a clue…

13

hello jaescobar.sanchez,

it’s F_GETFL for getting a flag, it’s reported message for blocking implementation
are you able to exclude the flag for opening the video node?

14

Hello JerryChang,

I tried to open the video node in all the possible ways, O_RDWR, O_RDONLY, with the flags, without the flags, etc. It makes no difference.

Thank you.

16

hello jaescobar.sanchez,

please also refer to VLC fails with L4T 32.5
we had 3rdparty application fails to open with USB camera on r32.5 release.

18

I have experienced a similar issue here.

In jetpack 4.5. ‘open’ returns different formats than ‘v4l2_open’.
In my case, ‘open’ returns the format provided in device tree while ‘v4l2_open’ only provides ISP output format and does not even list the raw camera format. Flags passed to v4l_open are ignored and I am therefore also unable to open more than one stream simultaneously using gthe new v4l argus pipeline.

Best regards,
Nico

19

UPDATE

besides the bug fixes,
we have also complete internal verification by including some code refinement.
you may download the pre-built binary for JetPack-4.5 / l4t-r32.5,
i.e. Topic168303_Apr21_libv4l2_nvargus.zip (101.1 KB)

keep the previously release version as below,
Topic168303_Mar15_libv4l2_nvargus.zip (101.1 KB)

it has include the fixes to make driver to get the media bus info.
you may replace /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so with the attachment,
please also perform a warm-reboot for confirmation.
thanks

1 Like
20

Hi @JerryChang ,

Unfortunately, that did not change anything in my tests. It appears that the returned formats depend on the combination of either ‘ioct’ or ‘v4l2_ioctl’ and the use of ‘V4L2_BUF_TYPE_VIDEO_CAPTURE’ or ‘V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE’

I would expect ‘V4L2_BUF_TYPE_VIDEO_CAPTURE’ and ‘V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE’ to return different formats dependeing on whether you would want to use single plane or multiplane capturing, but not ‘ioct’ or ‘v4l2_ioctl’

Here are my results when using the 4 possible combinations for a RAW10 RGGB camera and the following function to list formats (please change type and ioctl function accordingly)

void listFormats() const
{

	struct v4l2_fmtdesc fmtdesc;
    memset(&fmtdesc,0,sizeof(fmtdesc));
    fmtdesc.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	while (ioctl(fd,VIDIOC_ENUM_FMT,&fmtdesc) == 0)
	{   
		char* fourcc = (char*)&fmtdesc.pixelformat;

		printf("index  = %u\n", fmtdesc.index);
		printf("type   = %u\n", fmtdesc.type);
		printf("fourcc = %c%c%c%c\n", fourcc[0], fourcc[1], fourcc[2], fourcc[3]);
		printf("desc   = %s\n", fmtdesc.description);
		fmtdesc.index++;
	}
}

V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + v4l2_ioctl => returns expected result

index  = 0
type   = 9
fourcc = NM12
desc   = YUV 4:2:0

V4L2_BUF_TYPE_VIDEO_CAPTURE + v4l2_ioctl => returns multiplane format for single plane buffer type

index  = 0
type   = 1
fourcc = NM12
desc   = YUV 4:2:0

V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + ioctl => returns empty list

empty

V4L2_BUF_TYPE_VIDEO_CAPTURE + ioctl => returns expected RAW mode

index  = 0
type   = 1
fourcc = RG10
desc   = 10-bit Bayer RGRG/GBGB

Best regards,
Nico

21

Hello @JerryChang ,

With this file all is working again for me.

Thank you.

23

Hi Nico. I met the same issue like you,

I tried to open rtsp streaming using gstreamer command and control aelock using v4l2 api (or argusv4l2 api), and I don’t want this to block the streaming, which means the NONBLOCK flag need to work.

Do you have any solution for this?

Thanks in advance