ManagedMemory clarification

Hi there,

I wanted to know the bahaviour of function cuMemPrefetchAsync in one specific case, that is:

If I have one GPU and set a range of memory to be preferred on CPU via cuMemAdvise. I understand that when the device memory is oversubscribed, then this memory is likely to be evicted to CPU and the subsequent call to cuMemPrefetchAsync will bring the memory back to GPU.

It’s said in the doc

“By default, any mappings to the previous location of the migrated pages are removed and mappings for the new location are only setup on dstDevice. The exact behavior however also depends on the settings applied to this memory range via cuMemAdvise as described below”

But what’s unclear in the doc is, if the memory isn’t evicted. I’d assume if the memory is still resident on the device, trying to bring it back from the host will result in a nop. Calling prefetch won’t affect performance or result in undefined bahaviour.

Could you clariry? Thanks a lot for your help!

I assume you are using cudaMemAdviseSetPreferredLocation

The general guidelines for that are as follows:

  • Suggests which processor is the best location for data
  • Does not automatically cause migration
  • Data will be migrated to the preferred processor on-demand (or if prefetched)
  • If possible, data (P2P) mappings will be provided when other processors touch it
  • If mapping is not possible, data is migrated

calling cudaMemPrefetchAsync with a destination (target processor) that corresponds to the current location of that data doesn’t cause migration and is a no-op.

Thanks for your help Robert!