We’re trying to get a custom kernel module that interfaces with a PCIe FPGA card using a large reserved memory block working on the Xavier (JetPack 4.1, L4T 31.0.2). The same kernel module currently works on a TX2 (JetPack 3.2, L4T R28.2).
Patch posted in comment #2 in this thread is equivalent (of Jetson-Xavier) to the patch posted in comment #2 of the other thread. So, if the intention is to let PCIe IP access memory (reserved through device tree) bypassing SMMU, then this patch is still required.
Now, I’m wondering if the memory is really getting reserved? How did you confirm this?
Also, does this error occur when PCIe IP tries to access this reserved memory region? OR when CPU (in kernel context) tries to access the memory?
Looking at the error, it seems some kernel component is trying to access user space memory.
That’s only 779MB of additional reserved memory. While the TX2 emitted a warning at boot about overlapping reservations with this same block, there is no such warning on the Xavier. We’re not using the entire block yet, and we’ll deal with resizing or relocating (and eventually eliminating) it in the future.
Also, querying the running device tree confirms that the reserved block is present:
Thanks for the hint. This was definitely caused by some bad kernel module code that was trying to access user space memory directly. We were apparently getting away with it on both the TX2 and on a number of x86 systems, but not on the Xavier.
Fixing that bad kernel module code eliminates the error and makes the PCIe transfers through the reserved memory block work correctly.
This is most probably caused by the new ARM64 version being used in Xavier compared to TX2s, arm64 v8.2. TX2 has arm64 v8.
Since arm64 v8.1 a new cpufeature has been introduced called PAN (Privileged Access Never) for which kernel code has been updated to throw permission error when kernel tries to access userspace memory. I think this is the likely reason for the failures we are seeing.
We fixed it by doing copy_from_user() on all pointers potentially pointing to possible strings.