Cuda Device: Quadro FX 5800 and Tesla C1060 each with 4 GB of Memory
Toolkit Version: 3.2
SDK Version: 3.2
C++: MS Visual Studio 2008 SP1
Same configuration also running in Ubuntu 10.04 LTS x64
I need to visualize some big amount of volume data using a 3D texture. The volume right now has a size of 2.5 GB but
there will be more volumes of even bigger size in the near future to be visualized.
Using Ubuntu my application works as expected and i am able to use the full amount of memory the cuda devices provide.
Unfortunately under windows the wddm throws a monkey wrench in my plans…
Using nvidia-smi I was able to activate the compute only mode for the tesla cards and can now allocate the full amount
of memory on these bords. But i really need to be able to also use the full memory size of the quadro card.
Is it possible to bypass wddm on quadro or gforce boards and allocate more memory than wddm allows?
I know it is possible to allocate more memory in smaller pieces (eg. allocate 3 times 1 GB instead of allocating 1 time 3GB which wddm permits…)
so if i need 2.5 GB for the volume data. is it in any way possible to allocate this memory in smaller parts and after that map the texture reference
to this memory parts, having the tex refernce behave as normal?
Is the upper limit of allocating memory exactly the same in all cases (somehow defined in the driver) or is it different, depending on how many
device memory a specific card provides and, if this is the case, is it a fixed percental value or can i somehow check for the max value allowed in my code?
Hopefully someone here can help me out of this situation (and maybe even explain to me what the sense of this restriction of allocating memory is).
i was just wondering… Is it not possible what i was asking or is the question that stupid?
I would appreciate any advice or hint in this case…
I just need to find a way to use textures of more than 3 GByte on a non Tesla device under windows 7.
as some weeks passed since I last tried on the windows machine i am not
absolutely sure at the moment but my last note on one of my scribbling paper
says it was about 1798045696 bytes. so round about 1.6 GByte.
Ya,I’m quite sure that is the maximum value i reached…
[Edit]:
Here you can find the detailed list of tries. In another post i have made because
i didn’t know that it was related to the vista wddm on that time…
Like you, I’m trying to figure out what the “magic number” is for single memory allocations. Things wouldn’t be bad if we had a general formula so we were aware when developing or moving code to different cards.
I posted this in another thread, and I thought I was making sense, but in your case you have 12 GB of system memory and you were only able to do ~1.6 GB allocations. (Which makes what I said wrong). I really wish they would clarify how the allocation limit is decided and not black boxing everything.
Ok, so I’m really frustrated by the confusion so I did a little more digging.
It looks like there is a flag in the driver that sets the segment commmit limit, and there is also a global commit limmit. I’m thinking memory allocations are capped to whichever limit is lower.
Section of interest:
“The video memory manager will not allow the per segment nor the global commit limit to be violated. If a particular segment as a commit limit of 1 GB but the global commit limit is 256 MB, the video memory manager will not allow more than 256 MB of system memory to be mapped into that segment.”
Now the question still is… how do we know what those values are set to ahead of time?