External backup + storage setups for DGX Spark — what's actually working for you?

I’m running two DGX Sparks as a small cluster (ConnectX-7 link between them, nightly borg backups) and I’m about to buy external storage for a proper backup tier plus a model library. Before I do, I’d like to hear what other Spark owners are actually running day-to-day.

My planned setup:

  • TerraMaster D8 Hybrid over USB-C (USB 3.2 Gen2 10Gbps): 4x 3.5" SATA bays + 4x M.2 NVMe slots. Bays 1-2 in hardware RAID 1 with 2x IronWolf Pro 8TB as the borg target; one 4TB NVMe as a model library (GGUF/safetensors, vector DB snapshots); spare bays for a raw market-data mirror
  • Model library NFS-exported over the ConnectX-7 link so both Sparks share one store (same mount path + container bind-mounts on both nodes) instead of copying models to each node
  • Second Spark holds a lagged rsync replica of the borg repos over the same link
  • Small encrypted cloud repo for the irreplaceable subset

Questions for people with real hours on external storage:

  1. USB enclosure compatibility: anyone running a TerraMaster D8 Hybrid (or the newer D8 Hybrid 2 revision, or siblings like the D5 Hybrid / D4-320) on a Spark? Specifically: does the hardware RAID 1 volume present cleanly to DGX OS and survive reboots/power events without the array needing re-recognition? Do the M.2 slots and SATA bays both behave under UAS, and does SMART pass through (smartctl -d sat) for the drives behind the bridge? Any disconnects or link renegotiation under sustained transfers? If you tried a different multi-bay SATA box (QNAP TR-004, OWC, Sabrent) and it worked better or worse on the Spark, that’s exactly what I’m after.

  2. 24/7 stability: anyone running a USB DAS continuously attached for months? The Spark’s USB power/sleep behavior has some known oddities (e.g. the fans-stop-on-headless issue) — anything similar affecting attached storage?

  3. Hardware RAID vs mdadm: on DGX OS, are people trusting enclosure hardware RAID, or presenting individual disks and using mdadm? Any SMART passthrough issues with specific bridges?

  4. Model library placement: for those serving models from external NVMe (~1-2 GB/s over USB) vs internal — is external load speed acceptable in practice, or did you end up shuffling everything to the internal M.2 anyway? (I’ve seen reports that ASM2464-based NVMe enclosures behave well on the Spark’s 20Gbps ports while JHL7440-based ones underperform — confirmations welcome.)

  5. Alternatives I should consider: anyone doing this over the network instead (NAS via the 10GbE port, or storage hung off the ConnectX-7 link) and glad they did?

  6. Shared model store over the cluster link: anyone NFS-exporting a model library from one Spark and mounting it on the other (nconnect, or NFS-over-RDMA on the CX7) so containers on both nodes see one store? Any DGX OS wrinkles running nfs-kernel-server, or with mmap’d model loads over NFS? Most setups I’ve seen copy models to every node and I’d like to avoid that.

  7. NVMe-oF on the QSFP ports: has anyone exported NVMe-oF between Sparks (kernel nvmet target over RoCE on the CX7 link), or from a separate target box into a Spark’s spare QSFP port? Curious whether DGX OS ships the initiator/target modules ready to go and what real throughput looks like.

Not looking for general backup philosophy — specifically interested in hardware combos verified working (or failing) on a Spark. I’ll report back with whatever I end up building.