LapTime.run: a visual local LLM hardware simulator for comparing TTFT, decode speed, and model fit

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

Hi everyone,

I wanted to share a project I’ve been building called laptime.run (https://laptime.run).

It’s a local LLM hardware simulator meant to make hardware comparisons feel more intuitive and less like digging through disconnected benchmark tables. The idea is to help people answer questions like:

  • How different will this setup actually feel for interactive local AI use?
  • Where does time-to-first-token start to matter more than raw tokens/sec?
  • Which models realistically fit on a given system?
  • How do newer systems like DGX Spark / GB10-class machines compare with things like RTX 4090/5090 towers, Apple Silicon, and Strix Halo boxes?

A few things LapTime is trying to do differently:

  • Show “buyer feel” first: prompt ingest, TTFT, streamed response speed, and total lap time
  • Keep measured benchmark rows, modeled estimates, and community references clearly separated
  • Let you compare two setups side by side with the same model/workload/context
  • Make the assumptions visible instead of pretending every row is equally authoritative

Right now it includes:

  • NVIDIA GPU tiers
  • DGX Spark / GB10 systems
  • Apple Silicon systems
  • AMD Strix Halo systems
  • A growing local model catalog with fit indicators and source links

I’m especially interested in feedback from people here who are evaluating or using DGX Spark / GB10
systems for local inference.

A few questions I’d love input on:

  • Which local LLM workloads matter most for this class of machine?
  • What benchmark sources do you trust most for practical buying decisions?
  • What’s missing if you were using a tool like this to compare DGX Spark against a 4090/5090 tower or a
    big-memory Mac?
  • Would it be useful to add more NVIDIA-specific presets, models, or source references?

The project is here:

If people find it useful, I’d be happy to keep expanding the NVIDIA-side coverage and improve the assumptions with better source data.

Thanks, and I’d love any feedback!

2

I tried it out for a while, and here are my impressions.

I don’t have a DGX SPARK on hand right now, but about a month ago I used one for roughly a month. At that time, the performance of gpt-oss-120b was as follows:

docker run \
  --privileged \
  --gpus all \
  -it --rm \
  --network host \
  --ipc=host \
  -v $HOME/Downloads/vllm/models/:/models \
  vllm-node-mxfp4 \
  bash -c "vllm serve /models/gpt-oss-120b \
        --host 0.0.0.0 \
        --port 8000 \
        --enable-auto-tool-choice \
        --tool-call-parser openai \
        --reasoning-parser openai_gptoss \
        --gpu-memory-utilization 0.70 \
        --enable-prefix-caching \
        --load-format fastsafetensors \
        --quantization mxfp4 \
        --mxfp4-backend CUTLASS \
        --mxfp4-layers moe,qkv,o,lm_head \
        --attention-backend FLASHINFER \
        --kv-cache-dtype fp8 \
        --max-num-batched-tokens 8192"

| model                |   test |              t/s |     peak t/s |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:---------------------|-------:|-----------------:|-------------:|----------------:|----------------:|----------------:|
| /models/gpt-oss-120b |  pp512 | 1990.75 ± 121.62 |              |   234.52 ± 9.13 |   233.04 ± 9.13 |   283.00 ± 9.00 |
| /models/gpt-oss-120b |   tg32 |     62.84 ± 0.21 | 65.09 ± 0.22 |                 |                 |                 |
| /models/gpt-oss-120b |  pp512 | 2035.11 ± 109.96 |              |   236.59 ± 8.95 |   235.12 ± 8.95 |   285.22 ± 8.92 |
| /models/gpt-oss-120b |  tg128 |     62.89 ± 0.06 | 63.40 ± 0.49 |                 |                 |                 |
| /models/gpt-oss-120b | pp2048 | 4675.04 ± 205.09 |              |  407.94 ± 21.29 |  406.46 ± 21.29 |  457.61 ± 21.48 |
| /models/gpt-oss-120b |   tg32 |     61.99 ± 0.07 | 64.21 ± 0.07 |                 |                 |                 |
| /models/gpt-oss-120b | pp2048 | 4654.11 ± 109.56 |              |  383.15 ± 10.66 |  381.67 ± 10.66 |  432.59 ± 10.85 |
| /models/gpt-oss-120b |  tg128 |     62.23 ± 0.04 | 63.00 ± 0.00 |                 |                 |                 |
| /models/gpt-oss-120b | pp8192 |  6825.05 ± 15.11 |              | 1074.62 ± 13.78 | 1073.14 ± 13.78 | 1125.21 ± 13.30 |
| /models/gpt-oss-120b |   tg32 |     60.17 ± 0.16 | 62.33 ± 0.16 |                 |                 |                 |
| /models/gpt-oss-120b | pp8192 |  6809.99 ± 21.02 |              | 1068.75 ± 14.01 | 1067.27 ± 14.01 | 1119.86 ± 14.04 |
| /models/gpt-oss-120b |  tg128 |     60.36 ± 0.06 | 61.00 ± 0.00 |                 |                 |                 |

Meanwhile, the gpt-oss-120b-q4 numbers from laptime are:

  • Prefill: 168 tok/s
  • Decode: 3.2 tok/s
  • TTFT: 4814 ms
  • Total: 49.4 s
  • Prompt tokens: 180
  • Response tokens: 140

It’s obvious that the DGX Spark simulated performance in laptime is far below what I observed in my real-world testing.

3

Cho,

Thank you so much for this! I will investigate that and get it sorted out right away.

Tim

4

Cho,

I found the discrepancy and made some adjustments. LapTime had GPT OSS 120B modeled too pessimistically on DGX Spark because it was still treating it like a generic dense Q4 estimate scaled up from a smaller GB10 baseline. That breaks down for this model.

Your DGX Spark run used a tuned vLLM path with MXFP4 and backend-specific optimizations, while LapTime was extrapolating from an 8B-style baseline using simple size scaling. That led to a huge understatement of real DGX Spark performance, especially on decode and TTFT.

Here’s what I changed:

  • Changed GPT OSS 120B to an MXFP4 model with active-parameter-aware scaling instead of generic dense Q4
  • Added your DGX Spark result as a labeled community runtime row rather than leaving it as a pure estimate
  • Added red warning banners for estimated runs to make it more transparent when we don’t have real benchmark numbers and need to estimate the results.

So the short version is: the simulator published a bad heuristic for this case, and your real Spark
measurement exposed it. All the changes are now live and I’m continuing to hunt down more benchmark results so I can keep this thing honest!

Tim