According to unsloth documentation gpt-oss-120b training should fit in 65gb VRAM
```
Unsloth gpt-oss fine-tuning is 1.5x faster, uses 70% less VRAM, and supports 10x longer context lengths. gpt-oss-20b QLoRA training fits on a 14GB VRAM, and gpt-oss-120b works on 65GB VRAM.
```
I’m getting the following error:
```
Traceback (most recent call last):
File “/home/aeufemio/projects/dgx-spark-unsloth/./test_unsloth.py”, line 38, in
model, tokenizer = FastLanguageModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/unsloth/models/loader.py”, line 433, in from_pretrained
return FastModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/unsloth/models/loader.py”, line 991, in from_pretrained
model, tokenizer = FastBaseModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/unsloth/models/vision.py”, line 583, in from_pretrained
model = auto_model.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/transformers/models/auto/auto_factory.py”, line 604, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py”, line 277, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py”, line 5029, in from_pretrained
device_map = _get_device_map(model, device_map, max_memory, hf_quantizer, dtype, keep_in_fp32_regex)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py”, line 1365, in _get_device_map
hf_quantizer.validate_environment(device_map=device_map)
File “/usr/local/lib/python3.12/dist-packages/transformers/quantizers/quantizer_bnb_4bit.py”, line 127, in validate_environment
raise ValueError(
ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modules in 32-bit, you need to set llm_int8_enable_fp32_cpu_offload=True and pass a custom device_map to from_pretrained. Check Quantization for more details.
```