Call to function cuda-External-Memory-Get-Mapped-Buffer keeps failing on non-zero offsets

I’m trying to add the Optix denoiser to a Vulkan ray tracing application. I’ve been following the sample available here:

It works just fine if I allocate a single vk::DeviceMemory per vk::Buffer, but as soon as I allocate a single vk::DeviceMemory for multiple vk::Buffers, the memory offset is no longer 0 (assuming I’m not using the first block) and the call tho the cudaExternalMemoryGetMappedBuffer will fail with code 1.

If I run the program with cuda and optix disabled, both memory managers work correctly, so it’s not that the memory is malformed or corrupted.

Why does cuda crash when I try to use a memory object at an offset? Are there any additional steps I have to take in order to use it at an offset?

Here’s the segment of the code:

    vk::MemoryGetWin32HandleInfoKHR handleInfo;
	handleInfo.memory = cudaBuffer.buffer->m_allocId->memory();
	handleInfo.handleType = vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32;

	cudaBuffer.handle = VulkanContext::getDevice().getMemoryWin32HandleKHR(handleInfo);

	vk::MemoryRequirements memoryRequirements =

	cudaExternalMemoryHandleDesc cudaExtMemHandleDesc{};
	cudaExtMemHandleDesc.size = memoryRequirements.size;
	cudaExtMemHandleDesc.type = cudaExternalMemoryHandleTypeOpaqueWin32;
	cudaExtMemHandleDesc.handle.win32.handle = cudaBuffer.handle;

	cudaExternalMemory_t cudaExtMemVertexBuffer{};
	CUDA_CHECK(cudaImportExternalMemory(&cudaExtMemVertexBuffer, &cudaExtMemHandleDesc));

	cudaExternalMemoryBufferDesc cudaExtBufferDesc{};
	cudaExtBufferDesc.offset = cudaBuffer.buffer->m_allocId->offset;
	cudaExtBufferDesc.size = memoryRequirements.size;
	cudaExtBufferDesc.flags = 0;

If cudaExtBufferDesc.offset is not 0, the program will always fail at the final line.

How do I map to a memory object at an offset?