# NCCL For 2 Sparks Setup - Errors?

**URL:** <https://forums.developer.nvidia.com/t/nccl-for-2-sparks-setup-errors/353477>\
**Category:** DGX Spark / GB10\
**Tags:** spark\
**Created:** [December 4, 2025, 1:34am UTC](https://forums.developer.nvidia.com/t/nccl-for-2-sparks-setup-errors/353477 "2025-12-04T01:34:49Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![eugr](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/eugr/32/449615_2.png) [@eugr](https://forums.developer.nvidia.com/u/eugr)\
**Post date:** [December 4, 2025, 6:52pm UTC](https://forums.developer.nvidia.com/t/nccl-for-2-sparks-setup-errors/353477/7 "2025-12-04T18:52:26Z")

</div>

The playbook is incomplete, and if you set the variables the same way when using it for real workloads, you will lose a lot on latency, because it will be using Ethernet instead of Infiniband.

To get the most, you need to set these two in addition to the ones you already set:

```bash
export NCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1
export NCCL_IB_DISABLE=0

```

Use BOTH roceXXX devices that show as UP in `ibdev2netdev`. You will see NCCL test speeds increase to ~24 GB/s, but most importantly, the latency will be way down, and you will see real improvements in real workflows (e.g. VLLM, torch distributed, etc).

EDIT: to make sure it uses Infiniband, you can set `export NCCL_DEBUG=INFO` and read the logs.

---

_[View the full topic](https://forums.developer.nvidia.com/t/nccl-for-2-sparks-setup-errors/353477)._
