I am trying to build an internal memory pool which reserves a fixed size of memory from a fixed GPU device. It should support streamed allocator, and I do not want the memory pool to expand to avoid disturbing other processes on the same GPU. It seems to me that cudaMemPoolAttrReleaseThreshold is the way to go, but I am not sure if it is strict enough not to allow overflow of system memory ownship.
cudaMemPoolAttrReleaseThreshold does not prevent allocation of more memory.
In your memory pool implementation, you can keep track of the remaining free pool memory and simply throw an exception when more memory is requested than is available.
Do you have the option to use existing libraries? RMM GitHub - rapidsai/rmm: RAPIDS Memory Manager seems to have all you need. You can wrap an existing cudaMemPool_t in a cuda_async_view_memory_resource, then use a limiting_resource_adaptor to set an allocation limit.