As announced yesterday NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB has been released today. Designed for (RTX) Spark. Just in time for the end of my workday. 😎
NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB is a vision-language model (VLM) designed to run everyday agentic tasks locally, using a native 2-bit quantization to fit in 24GB of memory and run alongside other applications. It optimizes intelligence and speed per GB. It aims to achieve the optimal balance between speed and intelligence, being faster than most popular dense models while smarter than most popular MoE models with a similar memory footprint.
With detailed instructions for deployment with vLLM:
This could be an interesting ride regarding the published benchmarks.
Uh. There is also a second one that has been released:
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 is the quantized release of NVIDIA Nemotron 3.5 Super VL, a vision-language model for reasoning, coding, tool use, document understanding, and video understanding. It uses NVFP4 quantization to reduce memory requirements, with quantization-aware distillation from the BF16 model.
The model combines interleaved Mamba-2, Mixture-of-Experts (MoE), and attention layers with a C-RADIO vision encoder and a vision-language connector. It has 120B total parameters and 12B active parameters and supports context lengths up to 1M tokens. Multi-Token Prediction (MTP) enables native speculative decoding
Something for the weekend. 😅 ERR_TO_MANY_TO_TEST
.
Everything is open with complete provenance on both the data processing and training methods. There is a lot of value in having nemotron models as a jumping off point for custom LLMs.
asked it to check Spark GX10 cluster, in fact checking it’s own vllm logs. Lots of tool call errors.
Then one Simple task to rsync Redhat FP8 from worker to head, it synced from head to dev machine (where the harness is)!. Asked why and tell it to stop polling rsync progress: it rsync correctly but just kept polling every few seconds (before the adjust prompt , it polled every 2 or 3 minutes 😂) . Uncontrollable.
DS4 vision or Qwen FN did these kinds of job well. GLM flash is another level
@luisiam@cosinus, one thing the index doesn’t show: on one Spark, Super VL holds its whole 1,048,576-token window with room to spare (1.17 to 1.38 million tokens of KV cache on our box), and a code hidden at 10, 50 and 90 % of prompts up to 1,011,142 tokens was found 12 times out of 12. It also reads images and video. If you need neither the window nor the 120B, Bolt is the faster pick on the same box: 140 tok/s rewriting code with one request, about 200 tok/s in all at 16.
The way to choose is to run them on your own work, and that is easy on one box: dgx-spark-qwen38 serves Super VL, Bolt, Qwen3.8-Flash-Next and others on one Spark, one at a time behind the same OpenAI- and Anthropic-compatible endpoint, and its cockpit switches between them with Load: curl -fsSL https://raw.githubusercontent.com/hasso5703/dgx-spark-qwen38/main/get.sh | bash. Measurements and method: docs/nemotron-super-lane.md.
@tanbuikim7, on the tool call errors: which --tool-call-parser and --reasoning-parser did you run? With the card’s qwen3_coder and nemotron_v3, on one Spark, we got 20 of 20 tool calls right with the reasoning on and 20 of 20 with it off.
Context Size - good point. I will try both over the next few days for my usual tasks. Also because I’m curious how well a 2-bit quant can perform. I normally use FP8/4bit quants.
But first I will have to play a little with latest imagegen models:
The tool call is from hermes session, I saw many red error. It self corrected afterwards.
Besides, it does not strictly follow instructions and contracts, that’s what I need for unattended jobs. While flash versions ( Qwen FN , DSv4 or GLM 5.3 ) never miss my instructions.
Example in my orchestrator always ask to use custom agents only ; and subagents are not allowed to delegate to other agents. But Nemotron just simply call task then that task call another task.
Another example is in my last post, AGENTS.md listed machines clearly, but it just ignore and copy data to dev instead of head node. after it retried the synced hf cache was unusable. I need to manually resync.
Btw, nemotron super vl nvfp4 is fast in dual spark, with C2 at 120. I’m trying to apply in other use cases, not agentic coding/ server setup. The issue is my other hardware have 64GB vram or less, currently serving Qwen3.8 27b and Ornith 1.5 35b. They are doing the vision and document analysis quite well already.
@tanbuikim7 fair point, and a useful one. Our 20 of 20 is single tool calls checked one at a time through the engine’s API, with the reasoning on and off: it shows the parser and the format are right, not that the model keeps to rules through a long unattended session (no sub-agent delegating, an AGENTS.md it has to follow). We haven’t measured that, and your runs say more about it than our test does. If you still have the Hermes log with the red errors, I’d like to see what they were.
Good to know Super VL does 120 at C2 on two Sparks; we only run it on one. And for your 64 GB machines: everything we ship today is tuned for the 128 GB Spark, and a 64 GB setup is on our list.
