Hi all,
My system info:
OptiX Version:[6.1.2] Branch:[r421_00] Build Number:[26511982] ABI Version:[19] CUDA Version:[cuda100] 64-bit, Display driver: 430.26, Ubuntu 18.04 and gcc 7.3.0, 2x rtx 2080 Ti
I have been using very large buffers that I have to resize frequently. If I just use the C++ Buffer::setSize() I soon run into Out of memory exceptions. So it seems that the memory is not freed.
On the other hand, if I use destroy() and then create the buffer again with a new size, no exception is raised.
So, the question is: what happens with the memory when you resize a previous buffer with setSize()? Is it freed?
And, regarding performance, is it wise to just destroy and recreate buffers or should I try to reuse the buffer somehow?
I suspect what’s going on is that your resizes are triggering synchronous alloc, but asyncronous free, meaning that if you resize several times, you’ll allocate all the changes without deallocating any of the previous sizes (until later). This depends on what you’re doing and what the buffer type is.
What is the buffer type, and what are you storing in it? I.e., images or textures, variables, triangles, post processing buffers, something else?
When you say you’re using Buffer::setSize() frequently, are you calling it many times before launching, or many times in between two launches? Are the resizes generally increasing the size, or does it grow and shrink?
Performance-wise, the ideal scenario is to not resize buffers at all. It’s best to consolidate and limit all memory allocation and deallocation. If you have a maximum size you can reasonably set, such that you could safely never need any dynamic resizes after the first allocation, that would be the best case. Obviously, that doesn’t always work for everyone. The next best thing to do is to consolidate your resizes, meaning don’t try to resize to add 1 element at a time inside a loop that adds 100 elements, instead compute the size change you need first, and resize only once. If you’re doing many resizes in between two launches, doing that (consolidating your resizes into a single one) may resolve your out of memory issue.
I am simulating electromagnetic propagation and I use a RT_BUFFER_INPUT_OUTPUT buffer with RT_BUFFER_GPU_LOCAL, storing a custom type of around 32 bytes.
I compute the size of the buffers based on a number of things that may change between launches (like the number of receivers) and I call resize once between launches only when some of them have actually changed and it may grow or shrink the buffer. I compute the new size and then resize only once.
Following your advice I am going to change the memory management to try to avoid resizing unless it is absolutely necessary.
Hmmm, if you’re only resizing once between consecutive launches and running out of memory, that sounds potentially like a bug. All the memory allocation & deallocation should be resolved at launch time. Would it be easy to put together a small reproducer of the out of memory exception case, with either a code sample or an OptiX API Capture? (https://devtalk.nvidia.com/default/topic/803116/optix/error-with-rtpmodelupdate/post/4436953/#4436953)
I assume the buffer is never large enough that just having two copies active at the same time would not fit in your memory? How many resizes does it take before it crashes?
Actually I have spotted an error, as you said I was creating a buffer too large to fit the copies in the memory in the function I use to resize. It was correctly done in the one I use to create.
Anyway, thanks again for clarifying it. I am changing the memory management now.