cat /sys/block/zram*/comp_algorithm shows lzo
(sysctl vm.swappiness=65 (what’s almost distro default ==60)
It seems clearing zram devices on newer 5.x Kernels is much faster than experienced ~1-2MB/s (~10min for zram devices of 1GB zram_swapped, compressed memory pages) for swapoff command on 4.9.201-tegra (18.04.6 LTS).
Are there recommended adjustments for enabling faster swapoff?
(Thx@all)
Just a start note: Nanos are not designed for the JetPack 5.x. Are you speaking of a Nano? If not, then you might want to move this to the correct forum listed under here:
https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/70
When there you might indicate your use-case as to what you are actually trying to improve (a specific case). Not everything can run without swap, so be careful, but notice this command:
systemctl status swap.target
There are related commands like this:
# Permanent until enabled:
sudo systemctl disable swap.target
# Temporary, good for testing:
sudo systemctl stop swap.target
# Back on after disabled:
sudo systemctl enable swap.target
# Starting after stopping (temporary):
sudo systemctl start swap.target
It’s a Jetson Nano with GPU compute capability 5.3, JetPack ~4.5.1.
With care for swap dependent processes not being interrupted, what could delay/prevent some instruction fetching from dependent memory regions and did freeze systems few times with requiring reboot on other situations, Your suggestions improved swap/zram clearing timings with e.g.
systemctl stop swap.target
swapoff /dev/zram<?>
swapon /dev/zram<?>
systemctl start swap.targetfrom between relatively fast (almost comparable to some 5.x Kernels on other aarch64 device), ~1min up to ~6min for 1GB (but definitely less than ~10min/GB) for comparable demand on memory and cpus.
Yet, no obvious effects were recognized from system being reasoning for this endurance differences.
Thanks for Your suggestions
Not sure, the advice from @linuxdev may apply for recent L4T with JP5, but for your case with JP-4.5 you may also try disabling nvzramconfig service:
sudo systemctl status nvzramconfig.service
# If running you may stop it with:
sudo systemctl stop nvzramconfig.service
Possibly because swapoff is active only on one cpu core at a time and swap data is different compressed file sizes this could be reasons for varying swapoff delays.
With e.g.
sudo systemctl stop nvzramconfig.service
sudo systemctl stop swap.target
sudo swapoff /dev/zram<?>
...
clearing zram swap regions is ~2-2.7MB/s (~6min/GB).
Other suggestions for aarch64 (Rpi OS) were
sudo sysctl vm.vfs_cache_pressure=500
sudo vm.swappiness = 100
sudo sysctl vm.dirty_background_ratio=1
sudo vm.dirty_ratio = 50
sudo sysctl --system
(edit: ‘random’, slightly faster and slower results, influences on duration still unclear to me.)
Thanks for Your contribution.
If you don’t consider this solved, then you might want to provide a detailed description of your use-case. Basically, what is the driving motivation for doing this? Most people would not need to quickly turn off swap (swap would either be enabled or disabled with no change). Do you have some specific reason this matters? Is there something you need to solve?
From my knowledge this is a possibility for clearing zram memory regions, if swapped and compressed pages from programs or temporary files are not actively requested for a possibly longer period of time (hours to >days).
One example with need for clearing swap files (or compressed swap files) is when compiling finished with demand above available memory.
From my point of view, clearing zram is improved (with stopping swap.target and nvzramconfig.service before swapoff command), but still (mostly) a lot slower than on newer Kernels, therefore i would consider this still being “work in progress”.
Some details on parts of this might help to start…
Swap is “virtual” memory, meaning it is stored to a disk partition or file rather than in RAM. This is useful for user space programs, although it slows things down. Additionally, some devices cannot use virtual memory. The particular device most important on that list is the GPU, which can use only physical RAM. It is true though that if a user space program uses less RAM and instead swaps to virtual memory, then there is more available for the GPU. More precisely though, swap is block device memory.
zram is odd because it is RAM pretending to be a partition. Just calling it a “ramdisk” would imply no compression, whereas zram has compression on the RAM to make it more efficient. Anywhere a swap file or swap partition can be used, so too can zram. zram though is much faster. Sadly, it uses RAM! :) So when zram is used for swap it is an attempt to make swap much faster, and yet make the best use possible of the RAM. RAM pretending to be a disk used by a swap process which is trying to avoid using RAM. Confusing?
zram can be used as an ordinary partition, but you seem to be asking about the case of using it for swap.
If swap is off, then any zram used for swap will still be reserved, but there is no time required to unload it at swap off. If zram is used for swap and has content in it, then turning off swap requires first loading the content from zram back into ordinary RAM (when its identity changes from a block device back to physical memory). This is rather fast compared to turning off swap from a swap file or physical disk partition since RAM is much faster than eMMC or SD cards.
There is no need to ever clear swap files. If you’ve run into a case where you lack enough physical RAM for something requiring this, then turning off eMMC or SD card swap won’t help (e.g., needing more GPU memory). In fact, this would probably hurt. If turning off swap using zram, then initially it will hurt because the zram is still reserved even though user space processes have moved from that to ordinary RAM. If, following that, the zram partitions are themselves released, then all of that RAM becomes available to the GPU (or whatever wanted physical RAM). Keep in mind though that the processes which had previously swapped out to zram are now using regular RAM, and that RAM is not compressed. Depending on the content of the zram at the time of turning off swap and removing zram it might help or hurt what is available to the GPU.
I have doubts about the 4.x or 5.x kernel being a significant part of the difference in timing. I suspect what really matters is (A), what is in the zram, (B) if any swap is going to eMMC or SD, and (C) what overall RAM use changes occur while disabling.
What is it you want to speed up? If it is turning off swap, including zram, then I am skeptical that this will accomplish anything since it depends more on what is using the zram than the mechanism which turns it off.
for comparison: Kernel 6.2.x, aarch64 system, swapoff to a /dev/zramX device, comp_algorithm=[lz4hc], 400MB within ~1.8s, ~215MB/s
You’re speaking of a kernel with a couple of major releases newer than what the Nano can use. I suspect that the hardware performing the swapoff with a 6.2.x kernel is far faster/newer than the Nano. I suspect there is nothing you can do short of upgrading the hardware. Is there something you specifically need this to run faster for?