Hello everyone!
I have 2 GPUs: GeForce GTX 980 and GeForce GTX 650 Ti. Also I have 2 monitors, both are connected to one GPU - 650. But the command CudaMemGetInfo shows that 980 has free memory just 3,378 MBytes while it has to be near 4 GBytes. GPU-Z shows that 980 use only 3 Mbytes, but I can’t allocate memory on 980 more than 3,378 MBytes. I don’t understand - what could be the reason for this strange memory consumption? PhysX uses GTX 650…
If you run nvidia-smi -q you should see a list at the end that shows all the processes currently using the GPU. Unfortunately, with the default WDDM driver under Windows, you will not get information as to how much GPU memory each of those processes is using (since the memory is under control of Windows, not the CUDA driver). But you may be able to spot apps using the GPU that you did not expect to (e.g. browsers like Firefox or Internet Explorer, or background apps like Folding@Home).
Also note that GPU memory may be fragmented, which means you cannot allocate all remaining free memory in one single chunk.
here is output from “nvidia-smi -q”:
Driver Version : 384.76
Attached GPUs : 2
GPU 00000000:01:00.0
Product Name : GeForce GTX 980
...
FB Memory Usage
Total : 4096 MiB
Used : 35 MiB
Free : 4061 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
...
Processes : None
GPU 00000000:02:00.0
Product Name : GeForce GTX 650 Ti
...
FB Memory Usage
Total : 1024 MiB
Used : 329 MiB
Free : 695 MiB
BAR1 Memory Usage
Total : N/A
Used : N/A
Free : N/A
Compute Mode : Default
Utilization
Gpu : N/A
Memory : N/A
Encoder : N/A
Decoder : N/A
...
Processes : N/A
Not sure why “Processes” is showing up as N/A, on my system (64-bit Windows 7, Quadro K2200) it shows a list of the processes using the GPU. Some functionality of nvidia-smi is unavailable for consumer GPUs, but it seems difficult to imagine that the “Processes” tab would be one such feature. What OS are you using?
windows 10
is it possible that some context is always created on this card by any processes? but why it needs 700 MBytes?..
and command CuDevicePrimaryCtxRelease didn’t help to release the context
CUDA itself needs around 100 MB, so I can’t tell you where the memory goes with zero visibility via nvidia-smi. Are you accounting for possible fragmentation of the GPU memory by trying to allocate several small blocks, rather than one single big block? Have you tried rebooting the machine, in case there is a zombie process that is still holding on to GPU memory?
If you use fancy 3D features for your Windows desktop, that could eat up GPU memory, in addition to other possibilities already mentioned.
thanks for your time, njuffa.
I allocate memory on device with one bug array of ints. I was trying to search malwares and I turned off windows aero, but it don’t helps…
may be you know some utils that can test my memory? I mean, may be during installation GPU on motherboard I broke smth…
I don’t think there is any indication that anything is broken. There is just too little information present here to find out what the details of the situation are.
What I suggested above is to try allocating memory with smaller granularity, say in chunks of 100 MB, and then find how many chunks you can allocate. Possible fragmentation of the GPU memory means that remaining free memory may not be allocatable in one single chunk.
sorry, misunderstood
I did research in such way, I allocated with chunks exactly 100 MBytes, but 3.3 GBytes is my upperbound…