# Your setup doesn't support bf16/gpu

**URL:** <https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348>\
**Category:** DGX Spark / GB10\
**Tags:** ai-model-training, docker, spark, cuda\
**Created:** [January 3, 2026, 1:37am UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348 "2026-01-03T01:37:29Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![robs1978](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@robs1978](https://forums.developer.nvidia.com/u/robs1978)\
**Post date:** [January 3, 2026, 1:37am UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/1 "2026-01-03T01:37:29Z")

</div>

On my DGX Spark, when trying to run:

`docker run --gpus all --rm -it -v /home/rob/src/llm-training:/workspace -w /workspace ``nvcr.io/nvidia/pytorch:25.11-py3`` bash -c “pip install llamafactory && llamafactory-cli train configs/dapt_qwen3_4b.yaml”`

I get error:

`File “/usr/local/lib/python3.12/dist-packages/llamafactory/hparams/training_args.py”, line 105, in ` **`post_init`**  
`BaseTrainingArguments.` **`post_init`** `(self)`  
`File “/usr/local/lib/python3.12/dist-packages/transformers/training_args.py”, line 1747, in ` **`post_init`**  
`raise ValueError(error_message)`  
`ValueError: Your setup doesn’t support bf16/gpu.`

But the whole reason for the spark is to bf16, right?

Any help is appreciated. Thanks!

---

<div class="post-metadata">

**Author:** ![cyuen1](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/cyuen1/32/382649_2.png) [@cyuen1](https://forums.developer.nvidia.com/u/cyuen1)\
**Post date:** [January 5, 2026, 10:37pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/2 "2026-01-05T22:37:41Z")

</div>

Hi robs1978,

I’m seeing similar behavior with llama-factory and other models. Is your `dapt_qwen3_4b.yaml` a custom configuration, or did you get it from the official llama-factory repository?

---

<div class="post-metadata">

**Author:** ![bfurtaw](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bfurtaw/32/14047_2.png) [@bfurtaw](https://forums.developer.nvidia.com/u/bfurtaw)\
**Post date:** [January 9, 2026, 3:57pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/3 "2026-01-09T15:57:12Z")

</div>

> [@robs1978](#):
>
> docker run --gpus all --rm -it -v /home/rob/src/llm-training:/workspace -w /workspace `nvcr.io/nvidia/pytorch:25.11-py3` bash -c “pip install llamafactory &&

pip install llamafactory is installing a CPU only version of pytorch 2.9.1 which is causing the error, why do you need llamafactory?

Downloading torch-2.9.1-cp312-cp312-manylinux\_2\_28\_aarch64.whl (104.1 MB)

Run this command to diagnose the error, docker run --gpus all --rm -it [nvcr.io/nvidia/pytorch:25.11-py3](http://nvcr.io/nvidia/pytorch:25.11-py3) bash -c “pip install llamafactory && nvidia-smi && python -c ‘import torch; print(f"Torch version: {torch. **version** }“); print(f"BF16 supported: {torch.cuda.is\_bf16\_supported()}”)’”

I got these results,

Fri Jan 9 15:48:54 2026  
±----------------------------------------------------------------------------------------+  
| NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 |  
±----------------------------------------±-----------------------±---------------------+  
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |  
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |  
| | | MIG M. |  
|=========================================+========================+======================|  
| 0 NVIDIA GB10 On | 0000000F:01:00.0 Off | N/A |  
| N/A 44C P8 5W / N/A | Not Supported | 0% Default |  
| | | N/A |  
±----------------------------------------±-----------------------±---------------------+

±----------------------------------------------------------------------------------------+  
| Processes: |  
| GPU GI CI PID Type Process name GPU Memory |  
| ID ID Usage |  
|=========================================================================================|  
| No running processes found |  
±----------------------------------------------------------------------------------------+  
Torch version: 2.9.1+cpu  
BF16 supported: False

Torch included in the pytorch-25.11.py3 will not have this issue. It supports bf16. Llamafactory is overwriting torch installlation.

---

<div class="post-metadata">

**Author:** ![robs1978](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@robs1978](https://forums.developer.nvidia.com/u/robs1978)\
**Post date:** [January 11, 2026, 2:34am UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/4 "2026-01-11T02:34:42Z")

</div>

> [@bfurtaw](#):
>
> Run this command to diagnose the error, docker run --gpus all --rm -it [nvcr.io/nvidia/pytorch:25.11-py3](http://nvcr.io/nvidia/pytorch:25.11-py3) bash -c “pip install llamafactory && nvidia-smi && python -c ‘import torch; print(f"Torch version: {torch. **version** }“); print(f"BF16 supported: {torch.cuda.is\_bf16\_supported()}”)’”
> 
> I got these results,
> 
> Fri Jan 9 15:48:54 2026  
> ±----------------------------------------------------------------------------------------+  
> | NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 |  
> ±----------------------------------------±-----------------------±---------------------+  
> | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |  
> | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |  
> | | | MIG M. |  
> |=========================================+========================+======================|  
> | 0 NVIDIA GB10 On | 0000000F:01:00.0 Off | N/A |  
> | N/A 44C P8 5W / N/A | Not Supported | 0% Default |  
> | | | N/A |  
> ±----------------------------------------±-----------------------±---------------------+
> 
> ±----------------------------------------------------------------------------------------+  
> | Processes: |  
> | GPU GI CI PID Type Process name GPU Memory |  
> | ID ID Usage |  
> |=========================================================================================|  
> | No running processes found |  
> ±----------------------------------------------------------------------------------------+  
> Torch version: 2.9.1+cpu  
> BF16 supported: False
> 
> Torch included in the pytorch-25.11.py3 will not have this issue. It supports bf16. Llamafactory is overwriting torch installlation.

Aha. This makes sense. Because I had such a hard time getting GPU running without docker, I wasn’t thinking and didn’t realize it was already installed on the nvidia docker… I’ll give it a try soon.

---

<div class="post-metadata">

**Author:** ![robs1978](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@robs1978](https://forums.developer.nvidia.com/u/robs1978)\
**Post date:** [January 11, 2026, 8:45am UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/5 "2026-01-11T08:45:40Z")

</div>

Here is what finally worked for me:

`docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 `  
`–rm -it `  
`-e DISABLE_VERSION_CHECK=1 `  
`-v /path/to/your/project:/workspace `  
`-v ~/.cache/huggingface:/root/.cache/huggingface `  
`-w /workspace `  
`nvcr.io/nvidia/pytorch:25.11-py3`` `  
`bash -c "`  
`pip install llamafactory --no-deps && `  
`pip install torchaudio --no-deps && `  
`pip install datasets transformers accelerate peft trl bitsandbytes `  
`sentencepiece protobuf pyyaml packaging fire tyro omegaconf `  
`gradio uvicorn sse-starlette matplotlib scipy && `  
`llamafactory-cli train /workspace/your-config.yaml`  
`"`

The difference is --no-deps on llamafactory and torchaudio. This prevents pip from pulling in its own torch packages and breaking what’s already in the container.  
Also had to manually install the other dependencies since --no-deps skips them, and set DISABLE\_VERSION\_CHECK=1 because transformers 4.57.3 in the container is slightly newer than what LLaMA Factory’s version check allows (\<=4.57.1). Works fine (so far).  
Hope this saves someone else a few hours.

I have not completed a full SFT with this method yet. But at least it’s up and running right now.

---

<div class="post-metadata">

**Author:** ![bfurtaw](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bfurtaw/32/14047_2.png) [@bfurtaw](https://forums.developer.nvidia.com/u/bfurtaw)\
**Post date:** [January 12, 2026, 3:57pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/6 "2026-01-12T15:57:46Z")

</div>

How long is the SFT running? Are you using nvfp4 weights and activations, fastest on Spark.

---

<div class="post-metadata">

**Author:** ![robs1978](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@robs1978](https://forums.developer.nvidia.com/u/robs1978)\
**Post date:** [January 14, 2026, 11:55pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/7 "2026-01-14T23:55:58Z")

</div>

Thanks again for the reply.

**SFT duration:** The SFT that crashed was at step 4550 of 5520 (~82% complete), roughly 2 hours in. It was training on ~165K Q&A pairs. Crashed when running in local venv, worked fine after switching to the NVIDIA container.

**NVFP4:** Not using it - I’m doing LoRA fine-tuning with FP16. My understanding is NVFP4 is mainly for inference, not training? Happy to be corrected if there’s a training workflow that uses it.

**BF16:** Interesting - I assumed GB10 supports BF16 (Blackwell arch), but I got that error when using pip-installed PyTorch outside the container. Inside the NVIDIA container, I stuck with FP16 since it was working. Is BF16 actually supported on GB10, and is there a benefit over FP16 for training?

---

<div class="post-metadata">

**Author:** ![bfurtaw](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bfurtaw/32/14047_2.png) [@bfurtaw](https://forums.developer.nvidia.com/u/bfurtaw)\
**Post date:** [January 15, 2026, 3:23pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/8 "2026-01-15T15:23:56Z")

</div>

Only torch-2.10.0 cu130 and beyond are aware of the GB10 Micro-Architecture SM-121. I would recommend only training with SM-121.

I have heard some video applications have better quality inference at MXFP8, which is a no brainer. But I think training at BF16 is a standard.

When to Use Each

- BF16 for Training Stability: BF16 is the standard for training large models (LLMs) because its wide range prevents numerical “overflow” or “underflow” (where numbers become too large or too small to represent). This eliminates the need for complex “loss scaling” techniques required by FP16.
- FP16 for Inference & Efficiency: FP16 is often preferred for inference tasks where speed is critical and the range of values is already well-defined. On specific 2026-era benchmarks, FP16 has shown up to 3x faster evaluation speeds compared to BF16 during certain inference-heavy cycles.

---

<div class="post-metadata">

**Author:** ![robs1978](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@robs1978](https://forums.developer.nvidia.com/u/robs1978)\
**Post date:** [January 15, 2026, 10:26pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/9 "2026-01-15T22:26:25Z")

</div>

Thanks @bfurtaw! That explains it - I was hitting the BF16 error with pip-installed PyTorch. Switching to the NVIDIA container fixed it (likely has torch 2.10+ cu130).

For SFT timing: earlier test run was ~2hrs at step 4550/5520 when it crashed (venv issue, not container). Current DAPT run is processing ~1.7M chunks with LoRA on Qwen3-4B. Will report back with full timings.

I wish Nvidia would update DGX OS with pytorch, nvtop, etc that are compatible with SM-121.

I don’t know if I’m in the minority. For my specific project I bought the spark for, docker is just an extra layer.

Thanks again!

---

<div class="post-metadata">

**Author:** ![bfurtaw](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bfurtaw/32/14047_2.png) [@bfurtaw](https://forums.developer.nvidia.com/u/bfurtaw)\
**Post date:** [January 16, 2026, 4:19pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/10 "2026-01-16T16:19:23Z")

</div>

Right now free -h to determine free GPU memory. Watch buffer/cache should be around 1.2 GB. To drop page cache reserve , sudo sh -c ‘sync; echo 3 \> /proc/sys/vm/drop\_caches’ . Htop will watch memory usage. I started out using venv on DGX Spark but switched to conda for better management of pkg environ. Working on better Spark support for pytorch. Upgrade to CUDA 13.1 SDK much higher performance.

---

<div class="post-metadata">

**Author:** ![gergely1](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@gergely1](https://forums.developer.nvidia.com/u/gergely1)\
**Post date:** [January 16, 2026, 5:28pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/11 "2026-01-16T17:28:15Z")

</div>

I solved the original issue, after I created a new Python Environment and installed this pyTorch version:  
`pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu130`

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [January 30, 2026, 5:28pm UTC](https://forums.developer.nvidia.com/t/your-setup-doesnt-support-bf16-gpu/356348/12 "2026-01-30T17:28:49Z")

</div>

Closed on January 30, 2026, 5:28pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
