Has anyone tested this?
I am trying to get it to work with my DGX Spark and notebook with RTX 3060.
I’ll join you in a bit. Currently in a rabbit hole to optimize my stack. Please update with your findings. Thanks!
The playbook link: Install and Use NVIDIA PAIR | Playbooks
I built NVDC.ai (https://nvdc.ai) for this purpose and PAIR is a fraction of what NVDC already does. I’d much rather collaborate than run two parallel efforts, because NVDC is further along in a few areas:
- Verifiable nodes. Coordinator-verified nvTrust/NRAS attestation, so a client knows it’s talking to genuine NVIDIA silicon.
- Model fit per machine. Every model is marked Fits / Tight / Won’t fit against the node’s real memory budget (system RAM on unified-memory boxes like the Spark, VRAM on discrete GPUs). PAIR’s README currently leaves that to the user and the engine.
- Public/private switching. A node can serve only its owner or go live for the wider network, and falls back to local inference when the network is unreachable.
- Optional monetisation. Operators can set a price and be paid for work they serve, with signed receipts — or leave it off.
- A live public view of nodes and the models they’re serving - creating a decentralised inference marketplace with floating prices and an API for best ask.
Very interesting - as I also have been in the need for more compute power, I have started building something quite similar, but nonetheless with some other hopefully interesting aspects for our community, in particular to (hopefully) avoid international tax problems when sharing