I ran the Qwen3 235B Q3 precision in LM studio on DGX Spark, it was fine initially but then I noticed that the system would take up 20 GB of memory space doing nothing (or I believe nothing), and then I could not load the 103 GB size AI model due to lack of memory space.
Does any one know how to clear up that 20 GB? it seems out of nowhere. Thanks!
Please check out our FAQ on how to clear your cache DGX Spark / GB10 FAQ
In addition to the reply above, you may still not be able to run it as you need some RAM for KV cache and other buffers. You can try to clear caches and switch OS to multiuser (text only) mode to save some RAM on not loading the GUI, but it may still not be enough.
Whats the lowest memory before loading anything in spark ? And why cache is using such large memory ?
It’s a filesystem cache. Linux principle is free memory is wasted memory, so it aggressively caches stuff so it works faster. Normally it’s not a problem, but there is a bug in CUDA that doesn’t consider cache in free memory calculation. IOW, instead of using “available memory” it uses “free memory”.
As for lowest memory, in non-graphical mode the OS can take under 1GB. I’m not sure if CUDA memory allocation will have any safety margins on top of that though.
Is that always going to be the case? Are there any efforts to address it in cuda?
I noticed that vLLM containers will download a model causing most of the memory to appear in use. Then they fail to launch the model because the download filled the memory up and because vLLM check memory before launching.
I’m happy to empty the memory myself. But if I’m composing a few containers and bringing them up and down it’s quite clunky to need to manually flush the memory each time
I hope NVIDIA fixes it at some point.
For now, the workaround is to download models first, flush the caches, then run the models.
Not all apps are affected, llama.cpp works just fine without cache flush.
Thanks a lot, guys. Problem solved: as long as I clear the cache with sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches' right before loading the 256 B model with Q3 precision, it works fine with a 30k tokens context window. Good enough for me! Maybe a second DGX if funding secured LOL.
What kind of performance are you getting out of it?
2.5 tokens per second for the response.
My bad: 2.5 tps is from Hermes 70B. Qwen3 235B Q3 was 12 tps
I was expecting around 15 tps, but 12 is close enough.
Just ran unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q3_K_XL on my Spark and got 12.9 t/s on a simple image prompt. Slow, but usable.
I just posted about the RAM usage shown on the dashboard… It doesn’t seem to match up with any numbers in free.
I’m a linux noob so I may be missing something, but I posted a possible theory that they’re mixing up GB and Gi (because if you take the “available” column from free and subtract it from 128, you get their number… however the values in free are in Gi so the total is 119Gi not 128Gi).
FWIW I was trying to understand the cache memory buildup with Perplexity Enterprise Max’s Deep Research and to summarize it appears that the OS is seeing “unused system memory” as “idle wasted memory” and doesn't see the need to flush cache even after processes are concluded or containers are closed unless new processes are initiated that need system memory and it’ll drop the unused cache prioritizing the new processes.
I haven’t run experiments asides from AI Workbench that kept RAM at 70GB even after that training sequence was completed and containers stopped and before flushing cache through a reboot.
I’m planning on experimenting with running local persistent memory on the bundle and hope to find time throughout next week.
From what I can tell DGX Dashboard appears to not be accurate in reporting System Memory as a lot of the memory that is reported as being used in the dashboard is actually usable.
Running htop command in terminal is better at seeing true memory availability and utilization.
See my post two above yours - I think it’s not only that, but also that it’s mixing up GiB and GB, so the values are incorrect even taking cache/buffers into account. The lower the memory being used, the further out it seems to be.
I’ve been working on my own dashboard to use instead, which reads the values from /proc/meminfo and also includes some other stats (like CPU usage and temps). My plan is also to list Docker containers and their memory/CPU use with start/stop buttons.
their latest update has fixed the issue
Unfortunately it still doesn’t seem accurate - see here:
It’s not perfect, but for utilization monitoring is infinitely more reliable now compared to what it was before 😂
