Vulkan memoryHeaps and their memoryTypes

Gaming and Visualization Technologies APIs Vulkan
1
  1. Why does Vulkan report that I have 9 memoryTypes (7x propertyFlags=0, 1x propertyFlags=6, 1x propertyFlags=14) for my system Ram? What is the physical meaning of each memoryType? How to use each of these memoryType?
  2. Why are there 2 memory types for GPU RAM?
  3. Why are the total memory reported by Vulkan and cat /proc/meminfo different?

I recently posted the above question in https://stackoverflow.com/q/51624650/5722359. I was informed that:

  1. vkGetPhysicalDeviceMemoryProperties return multiple identical memory types because one is for VkBuffers, one for VkImages, and one per depth format (i.e. 7). Is this correct?
  2. It was speculated that one memoryType is for VkBuffers and another is for VkImage. Is this correct?
  3. It was speculated that fragmentation, other processes as well as the driver and my app need this memory hence Nvidia put a quota of 75% to report the memoryHeap size of the system RAM. Is this correct?
2

Cannot comment on 3, but the other 2 are pretty simple:

Memory HEAPS are actual physical types of memory. At the moment, NVIDIA has two, normal RAM and GPU V-RAM, AMD has 3 (an extra heap for the special on-chip memory that can be directly accessed by the CPU) and most integrated GPU (Intel, phones, etc.) have just one heap for the unified memory.

On the other hand, memory TYPES are fully artificial and have no direct physical meaning. Inspecting them by themselves is pretty pointless. Instead, they need to be inspected on a case-by-case basis for each allocation. Every image and buffer reports the types of memory it can be bound to as a bitmask. You AND the bitmask of all the objects together that you want to allocate in one device memory object, and then you can inspect the matching memory types. Then you match the wanted flags (device local or not, etc.) with the flags of each type and select based on that. The actual number of the type is pretty much irrelevant.

I have written this in another thread here, but any hardcoding of these types is the wrong approach. A match function based on the type bitmask for the objects that you want to bind to the memory as well as the desired flags is pretty much a must in every vulkan app. Since the types are required to be sorted by the standard this is fairly easy. Mine looks like this:

uint find_memory_type(const VkPhysicalDeviceMemoryProperties& mem_props, span<const VkMemoryPropertyFlags> propss, uint32 memory_type_bits)
{
	for(const auto props : propss)
		for(auto i = 0u; i != mem_props.memoryTypeCount; ++i)
			if(((1u << i) & memory_type_bits) != 0 && (mem_props.memoryTypes[i].propertyFlags & props) == props)
				return i;

	// We are out of device memory that meets the requested properties (that is, there was none to begin with)
	throw out_of_device_memory{ };
}

// Example: Get a type preferably on the GPU and CPU writable (like AMDs special ram), but also
//          be ok with it being just CPU writable
type = find_memory_type(props, {
	VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT | VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, // On the GPU and CPU writable
	VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT // CPU writable
}, bitmask);

Regards

3

@virtual_storm: You wrote that Nvidia has two types of heap, i.e. normal RAM and GPU V-RAM.

Clarification Questions:

  1. The Nvidia GPU (in my question) consist of VK_MEMORY_HEAP_DEVICE_LOCAL_BIT. How do I definitively know if it a normal RAM and GPU V-RAM through Vulkan or is it an information to be separately extracted from online specifications documentation of the GPU?
  2. Do you know why Nvidia defined only 2 memoryType and not 1, 3, 4, 5, 6,... etc VkMemoryType::propertyFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT for my GPU?
  3. Do you know if the Nvidia driver provided Vulkan the memoryTypes and memoryHeaps information on the System RAM, or did Vulkan extracted the information from the System or System RAM driver?

Thanks for sharing. I have a similar find_memory_type function too. Presently, I am curious to know the why device developer had allocated x number of memoryTypes for a specific and why the reported system RAM size do not match with its specification size.

4

  1. VK_MEMORY_HEAP_DEVICE_LOCAL_BIT signals that the heap is native to the GPU and the access is fast. Together with the type of the GPU from VkPhysicalDeviceProperties::deviceType, you can deduce what physical form the heap has. For VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU(every GPU that is connected via PCI-E or similar), the heap will be dedicated V-RAM on the GPU chip. For VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU (GPU as co-processor on the CPU, like Intel, AMD APU or most mobile GPUs) and VK_PHYSICAL_DEVICE_TYPE_CPU (simulators etc.) the device local heap will be the same as your system heap because these GPUs have no dedicated V-RAM. So yes, together with the device type you can know exactly what heap is what, no need to get a spec per GPU.

  2. Here is a blog post on GameWorks about the specifics of the NVIDIA memory types [url]https://developer.nvidia.com/what’s-your-vulkan-memory-type[/url]. Basically there are a bunch of special purpose types of system memory for render targets, buffers, textures etc in case there is not enough V-RAM and the engine wants to explicitly migrate resources to the CPU instead of leaving it to the driver/OS. The blog explicitly states that if you have a find_memory_type function then it works as intended. This is, as the blog post states, kind of intended for engine ports. Esp. since under Windows, you do not get a memory alloc fail and the OS migrates for you, i found it better to directly manage memory for assets depending on free V-RAM. Sadly Vulkan has no function to return free RAM, this is a major flaw and i hope to see an extension for that, but both NVIDIA and AMD have proprietary APIs to query free V-RAM.

  3. The memory TYPES are from the NVIDIA driver and may change without notice, the memory HEAPS are from the physical stats of your system. Vulkan is an API that the driver implements, if you are talking about the loader, no, the loader does not do anything. You can fully bypass the loader and interface with the driver directly. I would expect the NVIDIA driver to get the system RAM info directly from the OS, but i cannot know. AMD has open-source drivers for Vulkan on their GPUs, maybe a look at their code might help you? But of course NVIDIA might do something totally different.

The blog posts kind of states the reason why NVIDIA has all these extra memory types. As for the system RAM not matching under linux, i cannot help you, it matches under Win7 SP1 x64 GTX 970 driver 398.91.

Regards