Sdxl-customization fails - dual card system Titan RTX 24GB - RTX 3060 8GB

AI & Data Science NVIDIA AI Workbench
1

How can I find out why this project didn’t use my Titan RTX (Turing) instead of the smaller RTX 3050 (Ampere 8GB)

My desktop has

  1. RTX 3050 TI 8GB (Ampere) - id 1
  2. Titan RTX (Turing) - id 0

The project requires a GPU and appears to have tried to load the model into the 8GB card. How can I tell if it is because it needed a feature not on the Titan?

Loading pipeline components...:   0%|          | 0/7 [00:00<?, ?it/s]--- MODELS: Loading Model stabilityai/stable-diffusion-xl-base-1.0 ---

Loading pipeline components...:  29%|██▊       | 2/7 [00:00<00:00,  6.67it/s]
Loading pipeline components...:  57%|█████▋    | 4/7 [00:00<00:00,  8.08it/s]
Loading pipeline components...:  71%|███████▏  | 5/7 [00:00<00:00,  6.95it/s]
Loading pipeline components...: 100%|██████████| 7/7 [00:00<00:00,  9.52it/s]
Traceback (most recent call last):
 
   File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/home/workbench/.local/lib/python3.10/site-packages/torch/nn/parameter.py", line 68, in __deepcopy__
    self.data.clone(memory_format=torch.preserve_format), self.requires_grad
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 14.00 MiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Of the allocated memory 7.10 GiB is allocated by PyTorch, and 224.58 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
INFO:httpx:HTTP Request: GET https://checkip.amazonaws.com/ "HTTP/1.1 200 "
INFO:httpx:HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
--- MODELS: Configuring Pipe ---
--- MODELS: Model is ready for inference ---
http://localhost:8000
IMPORTANT: You are using gradio version 4.35.0, however version 4.44.1 is available, please upgrade.
--------


--------
2

I was able to run the project by removing the 8GB 3060 TI. The inference engine was able to create an image using just my Titan RTX card.

3

Hi, if this is the project sdxl-customization, it runs as a project container. This means you should be able to add an environment variable CUDA_VISIBLE_DEVICES=0 to explicitly use your Titan card. You can find this under Environment > Project Container > Variables, kindly let us know if this works for you.

Alternatively, if that doesn’t work setting NVIDIA_VISIBLE_DEVICES=0 on your host may work, how you do this will depend on your OS.

Docs: CUDA C++ Programming Guide — CUDA C++ Programming Guide
Alt docs: Specialized Configurations with Docker — NVIDIA Container Toolkit

1 Like
4

I should have known that and will check.

Pulling the card was easier after a couple spin cycles of troubleshooting. :-(

5

You may also want to set the Number of Desired GPUs to 2 so both are available to be filtered to the correct Titan one. This is under Environment > Project Container > Hardware.

6

I tried setting Number of Desired GPUs. That didn’t help on its own. I can l look at it it with the prioritization variable mentioned above.