In this presentation it is mentioned that “… our [NVIDIA Volta+] modern GPUs conform to C++ Parallel Forward Progress Guarantees” (quote from 29:33-29:44). How should this statement be interpreted?
I would interpret it as a guarantee that every thread that executed at least one instruction will execute another in a finite amount of time regardless of other work scheduled on the device, as long as that thread is not waiting on an explicit possibly blocking synchronization (__syncthreads, atomic wait, printf, …, but not atomic load/store/read-modify-write). But block preemption seems to break this, unless every preempted block must become resident again in a finite amount of time.