Pre-training Nanochat on DGX Spark (Standalone and Clustered mode)

ralph.lora · October 30, 2025, 2:54pm

I’m doing some tests on the spark to run pre-training for GitHub - karpathy/nanochat: The best ChatGPT that $100 can buy. · GitHub.
There’s a good discussion and a lot of shared benchmarks and instructions here:

raphael.amorim · December 14, 2025, 4:33pm

Lots of new things happening in that thread. Instructions to how to train it on 2 sparks:

gist.github.com

https://gist.github.com/emaadmanzoor/d245c0c0ce90b25b4d50c0ffc448f876

Train nanochat on 2 NVIDIA DGX Sparks.md

# Train nanochat on 2 NVIDIA DGX Sparks

*Last updated: November 3, 2025*

This guide is about training [nanochat](https://github.com/karpathy/nanochat/discussions/1) on 2 DGX Sparks
linked via QSFP/CX7. I estimate the training to take about 5 days (about half the training time on 1 DGX Spark).

## Prerequisites

Follow the following NVIDIA tutorials to link and test your Spark cluster:

This file has been truncated. show original

Topic		Replies	Views
Pretrain nanochat on 2 x DGX Sparks DGX Spark / GB10 Projects	3	624	January 2, 2026
Anyone got nanochat training working on the DGX spark? DGX Spark / GB10	10	1560	November 21, 2025
nvFP4 training - Playbook request DGX Spark / GB10	12	498	March 16, 2026
6x Spark setup DGX Spark / GB10	112	8584	April 25, 2026
Suggestion for https://build.nvidia.com/spark/ DGX Spark / GB10	2	530	February 18, 2026
DGX Spark PyTorch LLM training throughput up to 8x slower than expected DGX Spark / GB10	3	525	April 2, 2026
Should we as a community gofundme one Spark for Eugr's nightly builds? DGX Spark / GB10	52	1482	April 15, 2026
DGX Spark performance DGX Spark / GB10	50	4427	February 27, 2026
DGX Spark, Nemotron3, and NVFP4: Getting to 65+ tps DGX Spark / GB10 spark , nemotron , dgx	14	1896	December 22, 2025
DGX Spark Shutdown around 95°C during nanoChat Pretraining (20-30 min) DGX Spark / GB10	21	1427	March 23, 2026

Pre-training Nanochat on DGX Spark (Standalone and Clustered mode)

Related topics