I built an offline folder organizer on Qwen3.5 GGUF (llama.cpp through llama-cpp-python, served with model-compose) and wrote up a measured report. Here are the DGX Spark numbers next to the other machines I measured.
What it does
You list your projects one per line (tax-2026: receipts, invoices). The model picks a destination for each file, the code builds a plan, and nothing moves until you approve. Files are never deleted, and undo is one click.
9B Q4_K_M, 8K context, medians over the dev folders (~40 files each)
| DGX Spark | RTX 4090 | 16 GB M2 Mac | |
|---|---|---|---|
| Memory | 5.8 GB | 6.0 GB | 6.5 GB (6.9 GB asking four times) |
| Asking once | 14 s | 8 s | 2 min |
| Asking four times (list shuffled, majority vote) | 39 s | 20 s | 7 min 38 s |
- The Spark ran all 12 dev folders once each; the Mac times are from 4 of them.
- Plans on the Spark: asking four times was usable on 12 of 12 folders, asking once on 11 of 12. These folders were used to build the tool, so this checks that the Spark behaves like the 4090; it is not an accuracy result. The first plans matched the 4090’s in 7 of 12 folders (four asks) and 6 of 12 (one ask).
- Setup: model-compose 0.4.120 from PyPI (it does not pass
presence_penaltyandmin_p), llama-cpp-python built with CUDA. Memory is the server’s GPU memory fromnvidia-smi.
Accuracy (RTX 4090, 20 synthetic folders it had never seen)
- Given the project list with descriptions, 9B Q4 produced a usable plan (at most 1 line in 5 to fix) on 19/20
- When the model also had to write the project list: 6/20 (9B Q8)
- Smaller sizes, same setup: 0.8B (Q8) 2, 2B 9, 4B 1. Plain TF-IDF with no model: 16
- Caveat: the descriptions were written while looking at the answer key, so 19/20 is a best case
Report, setup and all tables:
The 4090 is about twice as fast here. If you have tuned llama.cpp on GB10, I’d like to hear which settings helped.