Second trigger for the illegal-access class, different from the Codex/DS4_SESSION_LAZY_GRAPH lead.
Setup: GB10 (sm_121a), driver 580.159.03, engine v0.5.0 (d9c8587) built make cuda-spark,
DeepSeek-V4-Flash-0731 IQ2XXS (self-built quant, 86.7 GiB) + DSpark-drafter-Q2K-Q8-0731.gguf,
-c 131072, DS4_SERVER_COALESCE_MAX=2, disk KV enabled with
--kv-disk-dir ~/.ds4/server-kv --kv-disk-space-mb 32768.
DS4_SESSION_LAZY_GRAPH=0 was already set when this fired, so whatever this is, it is not the
lazy-session-graph allocation. Client is an OpenAI-compatible agent (hermes-agent), not Codex, and it
rides /v1/chat/completions, not the Responses API.
The sequence, in order, from one crash:
- Tool-calling request arrives at ~59k tokens.
- The engine restores a 56,247-token bank from the disk KV tier.
ds4: cont admit rejected on comp-cache budget (bank 1: resident 439.0 MiB + need 36.6 MiB > budget 439.0 MiB)ds4: CUDA end commands failed: an illegal memory access was encountered
Then the usual aftermath you described: context poisoned, every later request 500s with
cuda prefill state reset failed, /v1/models still 200 so nothing outside notices.
What it looks like from here: the admission is rejected on the comp-cache budget, but the prefill
appears to proceed on that bank anyway โ i.e. a write into an extent that was never grown. The budget on
this box boots at budget=0.21 GiB because the weights plus the drafter fill unified memory, so the
reject path is hit constantly (measured over 20.6 h of uptime on v0.5.0: 1372 cont_admit_rejects,
1371 of 1401 requests ending up on the serial lane).
Kernel side: two Xid 13 (SM Warp Exception, Out Of Range Address) with a byte-identical ESR
across two separate crashes โ 0x1c81fb60. That determinism is why I do not think this is memory
pressure in the general sense.
Workaround that held: disabling the disk KV tier entirely. It is the only path that performs that
restore, so removing it removes the trigger. Cost is losing warm start across restarts.
Two things that did NOT help, in case they save someone else the time:
DS4_SESSION_LAZY_GRAPH=0 โ already on, crashed anyway.- Pinning
DS4_BATCH_VMM_BUDGET_MB=1536 โ it took cont admit rejected from 6 to 0 in a short
window, but a crash still occurred, and on v0.5.0 it cost about 1.3 GB of extra footprint
(grow-only pool). I am re-testing that pin now on v0.5.2 since trim-on-evict changes the calculus.
Happy to run any instrumented build against this โ the trigger is reproducible here in the sense that
it recurs under normal agentic load with disk KV on, though I have not reduced it to a single request.
Edit, cause i forgot the max reasoning:
Making reasoning_effort: "max" actually reach the model (two silent locks, one patch)
Following up on the Think Max discussion (@helgeโs hardmode TC-70-84 goes 60% โ 80% with "reasoning_effort":"max"). I went to turn it on and found it does not arrive โ and, more to the point, it does not tell you it did not arrive. There are two independent gates, both silent, and each one alone is enough to leave you at โhighโ while your client believes it asked for โmaxโ. Here is what they are, why the first one is safe to move, and a patch.
Lock 1: the context floor downgrades max โ high with no log line
ds4.c has #define DS4_THINK_MAX_MIN_CONTEXT 393216u, and ds4_think_mode_for_context() silently rewrites DS4_THINK_MAX to DS4_THINK_HIGH when the serverโs -c is below it. That is documented in --help (โThink Max is applied only when --ctx is at least 393216 tokensโ), so it is not hidden โ but nothing is logged at request time, so if you boot below 384K every max request comes back looking normal, just less thoughtful.
The part worth arguing about: this floor is policy, not a resource constraint. Think Max is nothing but a tokenized prompt prefix. The entire mechanism is one branch in the render path:
if (think_mode == DS4_THINK_MAX) {
bpe_tokenize_text(vocab, DS4_REASONING_EFFORT_MAX_PREFIX, out);
}
No buffer, no graph, no KV allocation is sized by the think mode. The floor encodes DeepSeekโs recommendation that Think Max wants room to deliberate, which is sound advice โ but on a single GB10 it is unreachable advice. At -c 262144 the context buffers alone are 5855 MiB and MemAvailable goes to zero (several of us hit that upthread); 384K is simply not a context this box can boot at while holding the weights. So the floorโs practical effect on 1ร Spark is not โprotect the user from a cramped reasoning budgetโ, it is โThink Max does not exist hereโ.
Setting it lower is not free โ a long deliberation can run past the context โ but that failure mode is the ordinary one the context handling already deals with, and it is visible when it happens. Being silently downgraded is not visible at all. I would rather have the sharp edge than the quiet one.
Lock 2: nothing in the request asks for max in the first place
Remove the floor and, for a lot of setups, nothing changes โ because no request ever carries the field. All four request parsers in ds4_server.c (parse_chat_request, parse_anthropic_request, parse_responses_request, parse_completion_request) initialize ds4_think_mode reasoning_effort = DS4_THINK_HIGH; and only move off it if the client sent reasoning_effort or output_config.effort.
That is correct behaviour, but it means Think Max is reachable only by clients that know the knob exists. Concretely, in my stack: hermes-agent has full reasoning_effort support including "max", but it only emits the field on its GitHub-Models and LM Studio provider paths โ for a generic OpenAI-compatible custom: provider (which is how everyone points hermes at ds4) it sends nothing. Same story for most agent CLIs: they were written against an API where the server default is the only reasoning setting there is. So the knob is there, the model supports it, and in practice it is never pressed.
The patch
Two env vars, both unset = current upstream behaviour exactly, both with a log line so you can see the config took effect. Against v0.5.2 (82d2a6f).
ds4.c โ make the floor overridable:
static uint32_t ds4_think_max_min_context_value(void) {
static uint32_t cached = 0;
if (cached == 0) {
cached = DS4_THINK_MAX_MIN_CONTEXT;
const char *e = getenv("DS4_THINK_MAX_MIN_CONTEXT");
if (e && e[0]) {
const long v = atol(e);
if (v > 0) {
cached = (uint32_t)v;
fprintf(stderr, "ds4: Think Max floor overridden: %u tokens (upstream default %u)\n",
cached, (unsigned)DS4_THINK_MAX_MIN_CONTEXT);
}
}
}
return cached;
}
ds4_think_mode ds4_think_mode_for_context(ds4_think_mode mode, int ctx_size) {
if (mode == DS4_THINK_MAX &&
(uint32_t)(ctx_size > 0 ? ctx_size : 0) < ds4_think_max_min_context_value()) {
return DS4_THINK_HIGH;
}
return mode;
}
ds4_server.c โ a server-side default effort, placed right after parse_reasoning_effort_name() so it can reuse the parser:
static ds4_think_mode ds4_server_default_think_effort(void) {
static int resolved = 0;
static ds4_think_mode cached = DS4_THINK_HIGH;
if (!resolved) {
resolved = 1;
const char *e = getenv("DS4_THINK_DEFAULT_EFFORT");
if (e && e[0]) {
ds4_think_mode m;
if (parse_reasoning_effort_name(e, &m)) {
cached = m;
fprintf(stderr, "ds4: default reasoning effort: %s\n", ds4_think_mode_name(cached));
} else {
fprintf(stderr, "ds4: DS4_THINK_DEFAULT_EFFORT='%s' not recognised, keeping high\n", e);
}
}
}
return cached;
}
then in each of the four parsers, replace ds4_think_mode reasoning_effort = DS4_THINK_HIGH; with ds4_think_mode reasoning_effort = ds4_server_default_think_effort();.
It is a default, not an override: a request that sends reasoning_effort still wins, including "none". The upstream unit test ds4_think_mode_for_context(DS4_THINK_MAX, 32768) == DS4_THINK_HIGH keeps passing, because with the env unset the value is still 393216.
Running it
DS4_THINK_MAX_MIN_CONTEXT=131072 DS4_THINK_DEFAULT_EFFORT=max \
ds4-server --cuda -m <model.gguf> --dspark <drafter.gguf> --host 0.0.0.0 --port 8000 -c 131072
Both lines show up in the log on the first request that touches them:
ds4: default reasoning effort: max
ds4: Think Max floor overridden: 131072 tokens (upstream default 393216)
To confirm it is actually in the prompt rather than just configured, send a trivial question and look at prompt_tokens: the max prefix is ~70 tokens, so a one-line user message lands around 94 instead of ~24. That is the cheapest end-to-end check I found โ the mode is not echoed in the response.
What it costs
Not as much as I expected, which surprised me. The prefix asks for exhaustive deliberation, but the model scales it to the task rather than padding everything: โwhat is the capital of Franceโ still came back in 23 completion tokens with a one-sentence reasoning_content. The cost shows up on hard problems, which is where you wanted it. One thing to watch on the client side: reasoning lands in reasoning_content and counts against the completion budget, so if your agent framework caps max_tokens low (mine defaults to 4096 on its mixture-of-agents path) a hard task can get truncated mid-thought. Raise that cap before blaming the model.
Setup for the numbers above: GB10, driver 580.159.03, v0.5.2 (82d2a6f) built make cuda-spark, DeepSeek-V4-Flash-0731 IQ2XXS (self-built, 86.7 GiB) + DSpark-drafter-Q2K-Q8-0731.gguf, -c 131072.
@entrpi โ if you would rather not ship a floor override, the second half stands on its own and is the one I would push for: a DS4_THINK_DEFAULT_EFFORT (or a --reasoning-effort flag) makes the setting reachable for every client that does not know the field exists, which today is most of them. Happy to send either half as a PR in whatever shape you prefer.