I am currently writing the part of a program that handles the organisation of a memory-heavy computation.
So I split the input data up into multiple blocks to be able to calculate everything.
This I do quite simply:
size_t absolute_free_memory[MAX_GPU_COUNT]; //contains amount of free_mem memory on GPU n
size_t IO_free_mem_memory[MAX_GPU_COUNT]; //contains the amount of memory usable by IO (input and output data arrays), thus (absolute_free_memory[n] - memory_for_calculation)
for (int j = 0; j < GPU_N; j++)
{
error = cudaSetDevice(j); //sets device to device j
printCudaError(error);
cudaMemGetInfo(&free_mem, &total); //gets memory info from it
absolute_free_mem_memory[j] = (int)free_mem; //puts it into mem info array
if (absolute_free_mem_memory[j] >= memory_for_calculation + IO_memory_for_calculation) //checks if there is enough free_mem mem to perform calculations
{
IO_free_mem_memory[j] = absolute_free_mem_memory[j] - memory_for_calculation * THREAD_COUNT * BLOCK_COUNT; //if so, IO_free_mem_memory = abs - mem_4_calc
}
else
{
printf("Not enough memory on GPU %i to perform calculations", j);
getchar();
return 0;
}
printf("GPU %i: %d MBytes abs\n", j, absolute_free_mem_memory[j]/ 1024 / 1024);
printf("GPU %i: %d MBytes io\n\n", j, IO_free_mem_memory[j]/ 1024 / 1024);
}
However, this gives me this output (note that I am using two RTX 2080Ti’s):
GPU 0: 1056 MBytes abs
GPU 0: 1040 MBytes io
GPU 1: 1056 MBytes abs
GPU 1: 1040 MBytes io
Which has to be incorrect, since there is no program running that uses that much VRAM, nor is any VRAM allocated beforehand. Also, later in the program, I do some memory readouts, to check whether the VRAM I want to allocate actually got allocated. There it correctly reports about 9GB, which seems to be correct.
With it reading lower free VRAM amounts than there actually should be, the code splits up the data into way more blocks than necessary, causing more IO and slowing it down. So this is a very unwanted problem.
Thanks for answering in advance!
You may have a problem with datatypes. Make sure you are using size_t everywhere (it’s impossible to tell from your code what type free_mem and total are. And why are you doing this:
absolute_free_mem_memory[j] = (int)free_mem; //puts it into mem info array
^^^^^
that is almost certainly wrong
If you still need help, make sure that every data type of every variable you use is clearly identified in your posting. Posting incomplete code is a good way to get your posting ignored.
Yes, that was the issue. I am not very experienced with coding nor asking questions about coding, since I have no training in it and am only 17 years old. Thank you for your answer!
I think a 17 year old or a 15 year old or probably a 12 year old that is actually writing code should be able to understand the concept of a complete code.
Can you compile it and run it without having to add anything or change anything? Then it is a complete code. If you’re not sure, pretend you are someone else looking at this question. Copy the code out of the question into a new file/new project, and see if you can compile it and run it without having to add anything or change anything.
It doesn’t have to be your whole code. It’s good practice when asking others for help to provide the shortest complete code, that still shows the problem you are having. This often requires some effort to create. However I don’t think the effort or understanding for that is beyond the level of a 17 year old either.
Not what I wanted to say but ok
It seems you have some experience in asking questions about coding, and in fact the idea of a complete code was already recited to you here:
https://stackoverflow.com/questions/56363869/cuda-in-c-how-to-fix-error-11-with-cudamemcpyasync
The point of reinforcing this idea (which I do quite a bit) is because I think it will make you more productive, and it will also improve the usefulness of the resultant dialog in a variety of ways. The reason SO has a stated rule like that is because through experience they have found that the question asking and answering process really works much better with that sort of rule in place.
We don’t have a corresponding set of published rules on these forums, but the underlying principle isn’t any different: you’re more likely to get useful help if you follow that principle.
This is a topic that is commonly pointed out:
https://stackoverflow.com/questions/57385066/how-to-create-a-persistent-framebuffer-in-cuda
https://stackoverflow.com/questions/57389063/cuda-reduce-sum-doesnt-work-when-image-is-large
Thats exactly what I find very discouraging about SO and other forums like these; No matter what I ask, it’s gonna be wrong in some way anyways. And thats what I mean with unexperienced; I still dont really know what to do but I always try to do it as well as possible. Here I may have simply forgot it. My excuses.