Best location for swap file on Xavier NX with a 250GB SSD

I need to install a swap file on my Xavier NX 8GB Developer Kit to improve training performance.

My Xavier NX boots up from a 128GB micro SD card into a 250GB SSD and is running JP5.0.2.

There is about 195GB of memory space left on the SSD so I have plenty of room for a large swap file.

The zram resides at /dev/zram0-5 on the SSD.

Dusty recommends this location for the swap file: sudo fallocate -l 4G /mnt/4GB.swap

AaStaLL recommends this location for the swap file: sudo fallocate -l 8G [/media/mySSD/swapfile]

So I am totally confused as to where the swap file should be created on the SSD and need some guidance as to where it should go.

Regards,
TCIII

Swap will help with user space processes. However, a GPU must use physical RAM. If swap lets user space processes give up physical RAM, then this could indirectly help the GPU, but it will not directly provide anything for the GPU. A Jetson (with an integrated/iGPU) shares its system RAM with the system, whereas a PC (with a discrete/dGPU) has its own RAM. One of the reasons people will pay for more RAM in the actual GPU (beyond what gaming would use, or even a CAD program) is because it is useful for AI training.

Should you add a swap file, then performance-wise, solid state memory won’t matter for location. An old style disk platter hard drive though would get better performance from cylinders near the outside of the platter (which is best for the operating system, and typically swap chooses the lower performance at the inner cylinders/end of the disk to allow the o/s to use outer cylinders). In the case of a swap file you just need to make sure it is available at the time swap enable is attempted.

zram is interesting. This is in fact physical RAM, but it is made to look like a block device (a hard drive or thumb drive, so on; loopback devices do this as well, but are based on files and not RAM). It is fast and won’t wear out solid state memory. However, the point of zram is not to about adding memory. It is about the format of how that memory is classified. Some programs must use swap space, and if you have enough RAM, then you get a performance boost when swapping to zram (the program thinks it is ordinary disk swap space, but it is actually much faster RAM). zram has a mount point, but it does not actually exist on any disk. That’s just where the drivers pretend to be a file or disk when software wants to talk to it.

/mnt is just a “traditional” place to mount temporary persistent storage (hard disks, thumb drives, SD cards, so on). It is owned by the system and so mounting there won’t cause conflicts when say a regular user does something to make the location unavailable since a regular user can’t modify /mnt.

The “/media” location I think is less desirable. The reason this shows up is that because when one is logged into a GUI and the user inserts temporary removable storage, e.g., thumb drive or an SD card to a USB reader, that auto mount will by tradition mount to “/media/<user's ID>” (this ID is “1000” for the first user added, and you might see use of either the user’s name or the numeric ID, e.g., “/media/nvidia” if the user name is “nvidia”, or “/media/1000”, or some subdirectory starting with “/media” and ending in that ID). This is less desirable because it implies the content is not permanently available.

If you were to remove the media containing the swap file while running, then the system would fail (badly). If you were to boot up with missing swap file space, then the result would depend on whether the swap was configured to always be on, or if it was configured such that it was allowed to fail. If configured as always on, then boot would stall there and never complete. You might end up having to either reinstall the system, or fix the problem by mounting the disk on another computer (which is easy with an SD card, but not so easy with eMMC memory soldered to the module).

Is this booting to the SSD as a root filesystem? Or is the SSD mounted somewhere as a secondary drive? If this is the entire root filesystem, then I’d suggest just make swap have its own unmistakable location, e.g., make directory “/swap”, and add a file there named “swapfile1.bin” (perhaps you want to add this binary swap content in steps and not as a single large file, thus you might name them in a series). For example:

  • sudo mkdir /swap
  • Then use the fallocate or other methods to create “/swap/swapfile1.bin” as an example name.

@linuxdev,

Thanks for the detailed response, much appreciated.

The Xavier NX is booting to the SSD as a root filesystem.

The SSD is nvme memory.

Dusty recommends turning the zram off: sudo systemctl disable nvzramconfig
Do you agree?

Regards,
TCIII

Dusty is actually one of the best experts you’ll find on anything Jetson (especially AI or CUDA), so if that is what he suggests, then yes. FYI, turning off zram makes more physical RAM available to the GPU, whereas adding virtual swap does nothing for the GPU unless some other process ends up using swap instead of physical memory. Perhaps a combination of both would give you the absolute maximum RAM available to GPU, but I’d first try to find out if just disabling zram gives you enough memory since (A) physical swap will be slower, and (B) physical swap will add to wear on any SSD or NVMe. There might be something in need of swap, but if it is not needed to add extra swap, then you are better off not enabling it in any form (zram is useful for more than swap though).

Incidentally, don’t forget to reboot after you disable nvzramconfig with systemctl.

1 Like

@linuxdev,

Thanks for the quick response, much appreciated.

I will give it consideration before installing the swap file.

Regards,
TCIII

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.