[Fixed] DGX Spark freezing and lockup issue; unable to load new models due to cache saturation

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1 
#!/bin/bash
THRESHOLD=40
CACHED_KB=$(awk '/^Cached:/ {print $2}' /proc/meminfo)
CACHED_GB=$(echo "scale=0; $CACHED_KB / 1024 / 1024" | bc)
IS_OVER=$(echo "$CACHED_GB > $THRESHOLD" | bc)
#echo $IS_OVER
#echo $CACHED_KB
#echo $CACHED_GB
echo "Önbellek boyutu ($CACHED_GB GB)"
if [ "$IS_OVER" -eq 1 ]; then
echo "The cache has exceeded the 40 GB limit. ($CACHED_GB GB). Flushing..."
sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
else
echo "The cache limit is not exceeded. ($CACHED_GB GB)."
fi

a bash script for cron
and I did a user.slice configuration
[Slice]
CPUQuota=90%
Everything is fine now.

2

Yeah, disk cache limiting memory for models is kinda annoying. Have to reset cache before every run of vllm with large models. I’m thinking about switching file system to ZFS from ext4 to configure maximum cache size. Other option is to switch to llama.cpp, seems it does allocates memory in more correct way than vllm.