Is there support for CONFIG_HUGETLBFS=y?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi,

I am wondering if there’s a support for CONFIG_HUGETLBFS and CONFIG_HUGETLB_PAGE? If I turn these options on, I get below error:

#error ARM Coherent DMA allocator does not (yet) support huge TLB

I see below code in kernel-4.9/arch/arm64/mm/dma-mapping.c:

#ifdef CONFIG_HUGETLB_PAGE
#error ARM Coherent DMA allocator does not (yet) support huge TLB
#endif

I wouldn’t want to just comment out this check as I am unsure of its effects overall on the system. But the reason I was thinking of turning on CONFIG_HUGETLBFS option was because now AGX Xavier has a 16GB of RAM which can be used for this.

2

Unfortunately, the ARM coherent DMA allocator in Linux kernel 4.9 does not support huge TLBs, so that won’t work.

3

Hi snarky,

I see that the files is L4T kernel is different than 4.9.108 kernel.org tree. In kernel.org tree there’s no code to error out if CONFIG_HUGETLB_PAGE is set. And that’s why I thought this could be NVIDIA tegra SOC specific.

4

It’s likely Tegra specific, perhaps because of the memory coherency requirements between the GPU and the four CPU clusters and other bus masters require additional work they haven’t managed to do yet.
Maybe they will fix it at some point, and maybe not. It very likely won’t work if you just remove the error. It also doesn’t work if you use plain 4.9 kernel, as it doesn’t have NVIDIA diffs for the Tegra specifics.
I’d love to hear more details from an NVIDIA person.

5

Right, hence I am hoping someone from NVIDIA responds on this thread.

Turning this option on is not a necessity but I wanted to see what happens since we have some unit tests to test some performance parameters with this on.

6

Have you tried enabling? I guess you can not enable it on ARM
We have never tried enabled CONFIG_HUGETLBFS and CONFIG_HUGETLB_PAGE on K4.9

https://tegra-sw-opengrok.nvidia.com/source/xref/stage-main/kernel/kernel-4.9/fs/Kconfig#200
189 config HUGETLBFS
190 bool “HugeTLB file system support”
191 depends on X86 || IA64 || SPARC64 || (S390 && 64BIT) ||
192 SYS_SUPPORTS_HUGETLBFS || BROKEN
193 help
194 hugetlbfs is a filesystem backing for HugeTLB pages, based on
195 ramfs. For architectures that support it, say Y here and read
196 <file:Documentation/vm/hugetlbpage.txt> for details.
197
198 If unsure, say N.
199
200 config HUGETLB_PAGE
201 def_bool HUGETLBFS
https://tegra-sw-opengrok.nvidia.com/source/xref/stage-main/kernel/kernel-4.9/Documentation/vm/hugetlbpage.txt

Looks like ARM doesn’t support this.

7

Hello bbasu,

Thank you for the response.

I can’t open the page on the link provided in the comment. But AFAIK, arm64 does support HUGETLBFS. I also see that CONFIG_SYS_SUPPORTS_HUGETLBFS is set to “=y”.

nvidia@magneto:~$ uname -a
Linux magneto 4.9.108-tegra #1 SMP PREEMPT Fri Sep 28 22:03:31 PDT 2018 aarch64 aarch64 aarch64 GNU/Linux
nvidia@magneto:~$ 
nvidia@magneto:~$ zcat /proc/config.gz | grep "CONFIG_SYS_SUPPORTS_HUGETLBFS=y"
CONFIG_SYS_SUPPORTS_HUGETLBFS=y

As I mentioned in my post, the error is occurring due to the pre-processor check that arch/arm64/mm/dma-mapping.c has. This check is not present in vanilla 4.9 kernel, that’s the reason I am wondering if this tegra specific check.

8

Hi Anish.aney,

Removing the pre-processor check would work in kernel-4.9.
Recently I was able to check HUGETLBFS feature with removing this check and enabling HUGETLBFS config.

I am able to observe the changes in Hugepage statistics i.e. HugePages_Total, HugePages_Free, HugePages_Rsvd are getting right values updated based on accesses and usages from the hugepage pool.

Thanks & Regards,
Sandipan

9

Thank you for the reply, spatra. I will remove the pre-processor check in the Xavier 4.9 tree and build the kernel.

10

Hi anish.aney

can you tell me how to remove the pre-processor check ?

thanks a lot