Environment:
- Container:
nvcr.io/nvidia/vllm:26.07-py3 - vLLM:
0.24.0+092c4842.nv26.7.59534043 - xgrammar:
0.2.0(as shipped) - Hardware: DGX Spark (GB10)
Symptom: Any request with tools/tool_choice set returns a 500, regardless of model:
{"error": {"message": "cannot import name 'normalize_tool_choice' from 'xgrammar'", "type": "InternalServerError", "code": 500}}
Requests without tools work fine — this only breaks tool-calling specifically.
Root cause: This container’s vLLM build calls xgrammar.normalize_tool_choice, which doesn’t exist in xgrammar==0.2.0 (the version pinned in the image) — it was added in xgrammar 0.2.4. So the container ships a vLLM build that’s outrun its own bundled dependency.
Workaround we verified works: pip install -U xgrammar inside the container bumps it to 0.2.4 and fixes the import — but pip’s resolver then silently downgrades transformers from the 5.6.1 this vLLM build actually requires down to 4.57.6 as a side effect, which would break other things. Re-pinning transformers==5.6.1 afterward keeps the import fixed. Built this into a two-line derived Dockerfile:
FROM nvcr.io/nvidia/vllm:26.07-py3
RUN pip install -q -U xgrammar && pip install -q transformers==5.6.1
With that, tool-calling works cleanly end-to-end — verified with real requests producing correct structured tool_calls.
One caveat I can’t fully vouch for: xgrammar==0.2.4 itself declares transformers<5,>=4.38.0 as a dependency constraint, so forcing transformers==5.6.1 back in is technically outside what xgrammar claims to support. Basic tool-calling checked out fine in our testing, but I haven’t exhaustively verified every xgrammar code path against transformers 5.x.
Ask: Could xgrammar be bumped to >=0.2.4 in the next 26.xx container build (with transformers re-verified against it), so this doesn’t need a manual patch?