TX1 -- swapon failed: Function not implemented

Hey, I’m trying to add swap to my TX1.

I’m running the newest jetpack.

I have tried mounting swap from a partition on a sata external and I’ve tried allocating a swapfile on flash and mounting that.

Both give me the following error.

swapon failed: Function not implemented

I’ve seen a couple places talking about how to add swap to the tk1.
I’ve also seen that this is probably a kernel option that needs to be enabled.

Do I have to recompile the kernel with this option or is there an easier/faster way?

Thanks,
Nathan

CONFIG_SWAP is not set in the TX1 defconfig, so I think you’ll have to rebuild the kernel. It’s not something that you can load as a module.

It looks like CONFIG_SWAP requires a built-in non-module feature (without this you cannot use swap). I’m going to guess it’s important enough that it makes its way back in at the next kernel release.

Under “make menuconfig” (or whatever your favorite config tool is), go into “General setup” and then look for “Support for paging of anonymous memory (swap)”.

It’s a bit of a chore to set up, but you’ll need both Aarch64 and gnueabihf/ARMv7 cross compile tool chains.

Ugh, looks like this is my project this weekend then.

I’ll post what I get after I finish.

Thanks,
Nathan

Recent posts from folks recompiling the kernel, see https://devtalk.nvidia.com/default/topic/894945/jetson-tx1/jetson-tx1/post/4741750/#4741750

I have the kernel (Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Tue May 17 16:29:05 PDT 2016 aarch64 aarch64 aarch64 GNU/Linux) and observe the message “swapon failed: Function not implemented” indicating that the kernel still needs to be compiled with CONFIG_SWAP.

Since TK1 with 32bit kernel is reported to support swap (I do not have TK1 access), is there a reason why CONFIG_SWAP is not enabled on the latest kernel? I am planning to re-compile the kernel and would like to understand what are the side-effects of enabling CONFIG_SWAP.

I am worried for the fact that the latest version “sudo dpkg --print-architecture” returns arm64* which means that the memory is loaded much more than previously loaded 32 bit applications. Under this assumption there has to be a reason for CONFIG_SWAP not to be enabled.

Cheers

  • I am not 100% confident about arm64, due to the fact that on the previous kernel I forced the dpkg to accept arm64.

I think swap on or off is just a preference in how someone expects to use the device. Could you elaborate on the question about “the memory is loaded much more than previously loaded 32 bit applications”?

Regarding 32-bit versus 64-bit on a JTX1: The kernel is 64-bit, it is the user space which is optionally 32-bit or 64-bit. The ARMv8-a 64-bit supports 64-bit and 32-bit, although not both at the same time (there is a separate CPU mode which can be switched in or out). The 32-bit can be referred to as ARMv8 (no “-a”), or alternatively aarch32 or arm32. The aarch32 accepts any of the older ARMv7 code, but the ARMv8 compiler might produce 32-bit code which is only “mostly” compatible for older pure ARMv7 systems (such as JTK1). The result is that anything ARMv7 will run on JTX1, but things compiled for JTX1 in 32-bit mode may not run on a JTK1 32-bit. 32-bit ARMv7 works on 32-bit aarch32.

The analogy that x86_64 desktop systems are 64-bit, but have 32-bit compatibility (e.g., i686) available is much the same on Jetson…a 64-bit system (where both CPU and user space are 64-bit) can have 32-bit compatibility as well. The package manager on either desktop systems or Jetson would need to be told it is ok to look for 32-bit versions of packages, as it wouldn’t know the two architectures are compatible without telling it so (plus 64-bit is preferable if both 32 and 64-bit are available). I have found flaws in this on Jetson though, there are some user space 32-bit armhf packages which should show up once the package manager is told of the compatibility, but it seems something in the package manager may not understand all of the subtleties. This is basically a package manager issue and not specific to Jetson. Keep in mind that there has been a lot of 32-bit ARMv7 hardware used out there, but 64-bit is very new.

The case of forcing 64-bit availability in the package manager when user space is 32-bit is invalid. If talking about cross compilers, a 32-bit user space Jetson could have a 32-bit compiler which outputs 64-bit code, but the compiler itself would still be 32-bit.

Before getting into side topics let me re-state my questions:

  1. Is there a reason why CONFIG_SWAP is not enabled on the kernel for TX1 but enabled on TK1?

  2. What are the side-effects of enabling CONFIG_SWAP while compiling the kernel for overall Jetson functionality?

Cheers

  1. I think that is just nobody thought to enable it…though I could be wrong.

  2. I enabled swap and have not noticed any instability related to it. The difficulty is that this must be an integrated feature, so both the Image and modules must be re-built with a new CONFIG_LOCALVERSION. So far I’ve only been able to build integrated features correctly using the version 4.8 compiler chain available with the driver documentation in the “baggage” subdirectory…all compiler versions newer than that end up failing on an illegal instruction (I’ve not figured out the exact code location it does this, but have tried Linaro 4.9, 5.2, and 5.3…the task would be much easier with a JTAG debugger, I do not believe kgdboc is suitable this early in the boot process for the debugging). This error does not depend on any particular feature that I know of.

We will enable CONFIG_SWAP in coming release, so you can experiment with swap space if you wish.

In the meantime, you can rebuild the kernel with CONFIG_SWAP support.

Thanks

Facing this problem with JetPack 3.1 on Jetson Tx1.
What is the workaround? How do I enable CONFIG_SWAP without having to rebuild the kernel?

Rebuilding the kernel is the only way. Swap is something which goes into the base image, not as a module (not every feature can be a module, swap is one of those). As long as you start with “/proc/config.gz” as the starting configuration, and set the CONFIG_LOCALVERSION to match the existing suffix of “uname -r”, you can enable swap and build the Image file and simply copy it over (though I suggest always using a second extlinux.conf boot entry and preserving the original Image file…rename your new image and point your new boot entry at this). Some changes to a base kernel configuration might mandate changing the CONFIG_LOCALVERSION and building modules again, but I believe adding swap does not require this.

@linuxdev

Thank you… I think this shall do.

I am also getting this error;

swapon failed: Function not implemented

below is the details on my env

L4T 28.1.0
Board: t210ref
Ubuntu 16.04 LTS
Kernel Version: 4.4.38-tegra

@kayccc mentioned that CONFIG_SWAP will be enabled in the coming release. I assume this did not happen? I’m working through this example

but I do not see where to set CONFIG_SWAP
Thanks for your help

CONFIG_SWAP did not make its way into R28.1, so you will need to build a kernel for this (you can’t do it as a module).

@jtfogar You can search for ‘swap’. The Option text is: Support for paging of anonymous memory (swap)

Hi, I am real confusing where to find that kernel configuration page. It seems like to open it by browser, is it? Could you please give me more details?

When you build a kernel you will need to set the configuration to match the current system, and then edit specific changes you want. Following this there are some kernel build targets which can be used to bring up different optional menus for editing that config (versus using an editor which might miss a dependency).

FYI, some of those use text mode (and are the ones I prefer). These require adding package “libncurses5-dev” to the computer used for compiling. You may also see references to cross-compile, which implies building on a PC host, but intended for the Jetson as a target (Jetson CPU architecture is not native to a PC host, and vice versa). You can ask more about that if needed, but here is some config information.

If you are at the kernel source tree and you run “make mrproper” the source will be turned completely pristine with no configuration. It isn’t a bad idea to start with this, but do beware that if you had any config in it this config will be gone. You can run “make clean” which won’t remove the config, but will clean out previous compiled files. You can “make tegra21_defconfig”, this will do a basic setup for a TX1…however, this may not match the kernel you are currently using. A running Jetson has a file “/proc/config.gz” which is a reflection of the current running system’s configuration…this is always the best starting spot if you have this. The file in the kernel source which is used is the one named “.config”…if you copy config.gz to that spot, then “gunzip config.gz” and “cp config .config” you will have a starting place which is a nearly exact match to the current system.

I say “nearly” because you still need to set “CONFIG_LOCALVERSION”. If you run the command “uname -r” on the Jetson it will respond with something like “4.4.38-tegra”. The “4.4.38” is from the kernel source, the suffix “-tegra” is from CONFIG_LOCALVERSION. You need to still match CONFIG_LOCALVERSION, and then you have an exact perfect match to the current system.

If you then run “make menuconfig”, or “make nconfig”, or “make xconfig”, you will see some of the different editor choices. My favorite is nconfig because it has both a search for symbol option (such as “localversion” or “swap”), plus you have an option to show even the options which can’t be selected due to missing prerequisites. I think the one you saw was from “make xconfig”.

Do beware that the “O=/somewhere” option is used to avoid cluttering up the original source and to do the building in a different directory (you will see references to this in cross compile guides). This option needs to be used at every step if you use it at all. Thus, “make nconfig” would be configuring the wrong directory if you use “O=/somewhere” in the commands which follow…use “make O=/some/where nconfig” or similar.

When you do finally install the resulting “Image” file don’t overwrite your old one, keep it handy. Perhaps change the file to something like “Image-4.4.38-tegra_custom”, and then add a duplicate boot entry in “/boot/extlinux/extlinx.conf”. You can use a serial console to select the alternate entry, or else set this entry to be the default. I hate setting to default without first testing, but this requires serial console to interact with U-Boot menu selection. See:
http://elinux.org/Jetson/TX1_Serial_Console
http://www.jetsonhacks.com/2015/12/01/serial-console-nvidia-jetson-tx1/

I’m working for a large customer of NVIDIA and we’re currently experiencing issues with our order of TX1s that we believe could be resolved by enabling swap. Would someone from NVIDIA please comment to provide some details of why a function that was enabled in R24.2 has been disabled for R28.1? Was there a technical reason for the removal? If not could we please ensure that this is enabled again in the next release?

Our use case is running deep learning models that get us quite close to the 4GB limit of the TX1, and without the ability to page out non-active memory to a swap file we are running into a number of issues (crashing, memory thrashing, fragmentation). Additionally, compilation on the TX1 has become a tougher task, as we have to limit ourselves to two compilation threads to avoid using too much memory.

I cannot comment for NVIDIA, but there is no technical reason to not enable swap. It is easy to enable though.