[V4L2] VIDIOC_REQBUFS cannot allocate more than 2GB of buffer

Hello,

I am currently trying to allocate a bit more than 2 GB of buffer via the VIDIOC_REQBUFS ioctl of V4L2 on a Jetson Xavier working with L4T 31.1

Our image size is 566442562 = 48 211 968 Bytes and we need to allocate 36 buffers for our application.

I noticed that the current maximum number of buffer is 32.

The dmesg output is the following

[  330.080662] Unable to handle kernel NULL pointer dereference at virtual address 00000020
[  330.088563] Mem abort info:
[  330.091547]   ESR = 0x96000005
[  330.094774]   Exception class = DABT (current EL), IL = 32 bits
[  330.100791]   SET = 0, FnV = 0
[  330.103876]   EA = 0, S1PTW = 0
[  330.107078] Data abort info:
[  330.109978]   ISV = 0, ISS = 0x00000005
[  330.113896]   CM = 0, WnR = 0
[  330.117055] user pgtable: 4k pages, 39-bit VAs, pgd = ffffffc3a2ea6000
[  330.123353] [0000000000000020] *pgd=0000000000000000, *pud=0000000000000000
[  330.130924] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[  330.136699] Modules linked in:
[  330.139630] CPU: 3 PID: 10671 Comm: user_process_camera_ Not tainted 4.9.108+ #124
[  330.146930] Hardware name: jetson-xavier (DT)
[  330.151394] task: ffffffc36b921c00 task.stack: ffffffc3ed81c000
[  330.157458] PC is at iommu_dma_alloc_iova+0xdc/0x130
[  330.162502] LR is at iommu_dma_alloc_iova+0xdc/0x130
[  330.167491] pc : [<ffffff80086d6a24>] lr : [<ffffff80086d6a24>] pstate: 80400145
[  330.174925] sp : ffffffc3ed81f870
[  330.178337] x29: ffffffc3ed81f880 x28: 0000000000000000
[  330.183849] x27: 000000000000003f x26: ffffffc3ec0a4fb8
[  330.189376] x25: 0000000000000000 x24: 0000000000000000
[  330.194977] x23: 0000000000000001 x22: 000000ffffffffff
[  330.200910] x21: ffffffc3ec347800 x20: 0000000000002dfb
[  330.206086] x19: 000000000000000c x18: 0000000000000010
[  330.211863] x17: 0000007fb7d40b10 x16: ffffff80082622c0
[  330.217376] x15: 0000000000000006 x14: 64656e67696c615f
[  330.223313] x13: 657a69732c32313d x12: 7466696873206666
[  330.229088] x11: 6666666666666666 x10: 203d74696d696c5f
[  330.234613] x9 : 00000000000007b8 x8 : 00000000000000c8
[  330.240677] x7 : ffffff8009ed4f88 x6 : ffffffc3ffdf6c30
[  330.246155] x5 : ffffffc370f65940 x4 : ffffffc3ffe03c80
[  330.251236] x3 : 0000000000008183 x2 : ffffffc370f65900
[  330.256825] x1 : 0000000000000009 x0 : 0000000000000000
[  330.261910]
[  330.263340] Process user_process_camera_ (pid: 10671, stack limit = 0xffffffc3ed81c028)
[  330.270481] Call trace:
[  330.272861] [<ffffff80086d6a24>] iommu_dma_alloc_iova+0xdc/0x130
[  330.278706] [<ffffff80086d6fe8>] iommu_dma_alloc+0x378/0x450
[  330.284053] [<ffffff80080a1e74>] __iommu_alloc_attrs+0xac/0x3f8
[  330.289685] [<ffffff8008b5e198>] vb2_dc_alloc+0x140/0x1a0
[  330.294720] [<ffffff8008b5728c>] __vb2_queue_alloc+0x15c/0x430
[  330.300316] [<ffffff8008b58410>] vb2_core_reqbufs+0x148/0x348
[  330.305397] [<ffffff8008b5b2c8>] vb2_ioctl_reqbufs+0x78/0xa8
[  330.310994] [<ffffff8008b3d378>] v4l_reqbufs+0x50/0x60
[  330.315546] [<ffffff8008b3bcec>] __video_do_ioctl+0x24c/0x2b8
[  330.321141] [<ffffff8008b3b694>] video_usercopy+0x2cc/0x688
[  330.326391] [<ffffff8008b3ba8c>] video_ioctl2+0x3c/0x50
[  330.331382] [<ffffff8008b362a0>] v4l2_ioctl+0xf8/0x128
[  330.336722] [<ffffff8008261a78>] do_vfs_ioctl+0xb0/0x8f8
[  330.342227] [<ffffff8008262354>] SyS_ioctl+0x94/0xa8
[  330.347228] [<ffffff8008083580>] el0_svc_naked+0x34/0x38
[  330.352393] ---[ end trace 733c07b2a206b1e2 ]---

How can we increase the DMA memory size to allocate more buffers ?

Thanks in advance,

Hi,
It is not clear if you run camera capture, video encoding, video decoding. We have samples at tegra_multimedia_api\samples. Please check these samples and make a patch on either one so that we can try to reproduce it.

We have max buffer number definition in all cases. 32 is a very large number and should exceed limitation of all cases.

Hi,

Thank you for your answer.

I am running raw camera capture with the V4L2 driver API. I am not using the multimedia api.

I am trying to allocate 9 buffers of 50MB for 4 sensors through the V4L2 ioctl VIDIOC_REQBUFS in four different threads:

fd = open("/dev/videoX", O_RDWR);
	if (fd == -1)
	{
		perror("Opening video device");
	}

	struct v4l2_requestbuffers req = { 0 };
	req.count = 9;
	req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	req.memory = V4L2_MEMORY_MMAP;
 
	if (-1 == ioctl(fd, VIDIOC_REQBUFS, &req))
	{
		perror("Requesting Buffer");
	}

With X going from 0 to 3 in “/dev/videoX”.

I am also trying to modify the file hardware\nvidia\soc\t19x\kernel-dts\tegra194-soc\tegra194-soc-base.dtsi and more specifically the following node:

common_as: common {
        iova-start = <0x0 0x80000000>;
        iova-size = <0x0 0x7FF00000>;
	alignment = <0xFFFFF>;
	num-pf-page = <0>;
	gap-page = <1>;
};

I tried to modify the parameter iova-size from 0x7FF00000 to 0xFFF00000.
This allows me to increase the number of allocated buffer but I have runtime issues as follow:

[ 1184.872342] mc-err: vpr base=0:0, size=0, ctrl=1, override:(a01a8340, fcee10c1, 1, 0)
[ 1184.872545] mc-err: (255) csw_viw: MC request violates VPR requirements
[ 1184.872666] mc-err:   status = 0x0ff74072; addr = 0xffffffff00; hi_adr_reg=008
[ 1184.872835] mc-err:   secure: yes, access-type: write
[ 1184.872970] mc-err: vpr base=0:0, size=0, ctrl=1, override:(a01a8340, fcee10c1, 1, 0)
[ 1184.873114] mc-err: (255) csw_viw: MC request violates VPR requirements
[ 1184.873231] mc-err:   status = 0x0ff74072; addr = 0xffffffff00; hi_adr_reg=008
[ 1184.873356] mc-err:   secure: yes, access-type: write
[ 1184.873485] mc-err: vpr base=0:0, size=0, ctrl=1, override:(a01a8340, fcee10c1, 1, 0)
[ 1184.873615] mc-err: (255) csw_viw: MC request violates VPR requirements
[ 1184.873730] mc-err:   status = 0x0ff74072; addr = 0xffffffff00; hi_adr_reg=008
[ 1184.874122] mc-err:   secure: yes, access-type: write
[ 1184.874551] mc-err: vpr base=0:0, size=0, ctrl=1, override:(a01a8340, fcee10c1, 1, 0)
[ 1184.875243] mc-err: (255) csw_viw: MC request violates VPR requirements
[ 1184.876156] mc-err:   status = 0x0ff74072; addr = 0xffffffff00; hi_adr_reg=008
[ 1184.883787] mc-err:   secure: yes, access-type: write
[ 1184.888767] mc-err: Too many MC errors; throttling prints
[ 1184.894412] __arm_smmu_context_fault: 141 callbacks suppressed
[ 1184.894436] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x7c000000, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.068142] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x7c100800, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.071103] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x7804cc00, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.072287] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x74000000, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.072650] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x740174c0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.073026] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x7402dbc0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.073458] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x74045500, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.073808] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x7405cec0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.074156] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x74071ac0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[ 1185.268050] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x740f8000, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0

Do you have any idea what else should I do to increase the iova-size properly ?

Thanks in advance,

Hi,
We are checking if the setting can be customized. Will update.

Hi,
common_as is general setting to multiple functions. Please try to ddd pixel_as specific to isp and vi functions.

isp_domain {
    address-space = <&<b>pixel_as</b>>;
    sid-list = <TEGRA_SID_ISP>;
};
(...)
vi_domain {
    address-space = <&<b>pixel_as</b>>;
    sid-list = <TEGRA_SID_VI>;
};
(...)
pixel_as: pixel {
    iova-start = <0x0 0x80000000>;
    iova-size = <0x4 0x3FFFFFFF>;
    alignment = <0xFFFFF>;
    num-pf-page = <0>;
    gap-page = <1>;
};

Hi,

Thanks for your answer.

I tried your patch and I can increase the number of buffers but I still have some errors:

[  523.541184] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x64000000, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.541705] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x6401e800, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.542055] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x64033f80, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.542475] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x64049e40, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.542823] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x64061f40, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.543159] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x64076cc0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.544075] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x640a9740, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.552381] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x642a00c0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.566177] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x645ddec0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0
[  523.579841] t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu2, iova=0x6490e3c0, fsynr=0x11, cb=22, sid=4(0x4 - VI), pgd=0, pud=0, pmd=0, pte=0

I got only one mc-err which is far less than before and maybe not related:

[  361.921128] mc-err: vpr base=0:0, size=0, ctrl=1, override:(a01a8340, fcee10c1, 1, 0)
[  361.921139] mc-err: (255) csw_viw: MC request violates VPR requirements
[  361.921148] mc-err:   status = 0x0ff74072; addr = 0xffffffff00; hi_adr_reg=008
[  361.921153] mc-err:   secure: yes, access-type: write

The images I get from the V4L2 API are all empty.

Do I have to specify elsewhere the memory increase allocated to the IOVA ?
It looks like the memory management unit is not fully aware of the IOVA increased size ?

Thanks in advance,

Hi,
If your sensors are hard linked to Xavier CSI ports, you may check [Sensor Driver Programming Guide] to modify device tree accordingly.

Hi,

The [Sensor Driver Programming Guide] does not talk about memory allocation.

It seems that the issue is more a Linux kernel issue than a sensor issue. I would like to increase the DMA zone area and with your patch I managed to do that but the rest of the kernel seems to be unaware of this modification hence the unhandled context fault.

How could I increase the DMA zone area on the Jetson Xavier ?

Thanks in advance,

Hi,
Is your sensor USB camera, YUV sensor via Xavier CSI port, or Bayer sensor via Xavier CSI port?

Hi,

The sensor is a 10-bit RAW sensor connected via Xavier CSI port.

It’s the OV24A1B sensor from omnivision.

Kind regards,

Hi,
Please try r32.1.
r31.1 is developer preview and we have enhancement in r32.1.