We are currently using the Jetson Orin to run our pipeline code that basically decodes multiple camera streams using Deepstream and runs some AI models on the GPU and then runs post processing business logic on the CPU. The issue is the device can run only a few streams with limited physical + Zram post which it crashes due to running out of memory.
Hence, we are exploring adding swap which has helped to increase the capability to run more streams. I understand that adding SWAP can cause wear and tear of the SSD in a long run. However, if we keep that aside for a while, is there an optimal SWAP for Orin that should be added beyond which it is not recommended. Something similar to the logic behind adding 3.6G of Zram which is half of 7.2G of physical RAM that comes with the device.
I will suggest that this is not as straightforward of a question as it may seem. The GPU itself can only use physical RAM. If this were a discrete GPU (dGPU) on the PCI bus, then the GPU would have its own RAM; this is an integrated GPU (iGPU), and it shares system RAM. Only physical RAM works with this GPU. Virtual memory would not be useful.
There are also limitations on fragmented memory. The GPU will likely require contiguous blocks of memory. Sometimes a kernel argument can reserve a particular block of RAM for the GPU as the kernel loads. This prevents other parts of the system from using pieces of that block and fragmenting it.
Swap can be useful for user space programs, which indirectly makes more physical RAM available, but unless your lack of RAM is from user space, and not GPU memory, it likely won’t help.
I could be wrong, but I don’t think adding swap will help other than to some minor extent.
Thanks for your reply. As i mentioned, i tried running with swap and it actually helped with user space programs. Also, I am not looking for more GPU RAM. Its able to handle decoding of multiple streams.
What i am looking for here is a recommendation on how to decide the optimal swap size. Beyond which there would be issues with the device. Right now I augmented the zram with 8GB of swap memory but what is the max limit of swap i can go to?
If you are swapping to disk, and not compressed RAM, then the old rule of thumb was to use a swap partition less than or equal to twice the physical RAM. In reality though, you might consider adding multiple swap files, each being minimal, and then enabling more if needed. When you find the total swap that works for you, then you have your answer. It is possible that performance will suffer. I don’t think there is a limit on the amount of swap, at least not a limit that would apply with most systems (for example, you could probably turn an entire NVMe or m.2 drive into swap…it might be terrible performance, but I doubt you’d run into a limitation before some impractically huge swap size since it is 64-bit).