On similar machines, one with Windows 8.1 and one with Windows 10 I noticed that code developed on the Windows 8.1 PC (CUDA 8.0 2 GTX 1080ti GPUs, Visual Studio 2015) would run fine using about 10 GB of 11 GB available. The same application run on a Windows 10 PC with the same configuration would crash with out of memory errors until I reduced the memory footprint down to 9 GB per GPU.
I was ready to return the GPUs thinking I had some ‘lemons’ until I realized this was a known issue.
Does not appear to NVIDIA’s fault, and I have little hope that MSFT will do anything about this issue. There also are some other problems with NVVP in Windows 10, but I will complain about those later.
Before I switch back to Windows 8.1 is there any possible ‘fix’ to this issue?
For example is it possible to ‘force’ WDDM driver mode 1.0 while still using Windows 10?
What a pain, I painstakingly mapped out all my memory usage to no avail…
Have to use Windows for this project so back to Windows 8.1, which has a worse GUI but at least has been stable for these large applications. Already bought and installed Win 10 and wonder if I can downgrade to 8.1 without buying another OS.
I wonder if TCC driver mode has the same issue, will check on one of my work PCs Monday.
One would think that given the huge increase in those using NVIDIA GPU for Deep Learning etc MSFT would pay attention to such issues.
Scaling those numbers for an 11 GB card, it does look like same percentage of overhead: 4.9 GB / 6 * 11 = 8.983 GB, i.e. the 9 GB you are observing. In other words: 95% of GPU memory available under Windows 7, and 81% available under Windows 10.
In the thread you linked, they observed similar numbers as well: “VRAM on card 6144 MB. The available VRAM was still around 5978 MB ! [on Win7, NJ] Now with windows 10 we only have around ~5000 MB”. That’s 97.3% and 81.3%, respectively.
Similarly, this user on the same thread: “I recently moved from windows 7 to windows 10, and have the same issue. Under windows 7 I was able to use 95% of the GPU memory for running Theano in deep learning applications, by setting cnmem=1.0, under Windows 10 I am limited to cnmem=0.8, which is equal to 80% of the available GPU memory (3.2 of 4 GB, Nvidia GTX 960).”
I would hazard a guess that not much of the GPU-accelerated AI work happens on Windows.