My question relates to how CUDA streams are mapped to the hardware.
Streams execute in-order and each successive operation is guaranteed that the previous operation is finished, i.e. two kernel launches into the same stream have the guarantee that the second kernel sees all changes made by the first kernel.
Recently I have worked with Vulkan quite a bit and came across the concept of queues, which I mapped to the same functionality in mind, until I learned that commands in a Vulkan queue “may” execute in parallel and synchronization must be manually enforced with barriers, i.e. they are not streams.
How does all of that map to the hardware?
When I launch “vulkaninfo”, it reports back (for my Titan V) that it has 16 queues which support all flags (typically called graphics queues), 8 which are primarily “compute” and some other transfer queues.
Is my understanding correct that these “hardware queues” just specify a launch order but the behaviour that is exhibited in streams (operations are serialized) is explicitly enforced by the driver, i.e. the driver inserts sync points/barriers in between commands automatically when issuing them to the same stream with CUDA, but with Vulkan the queue just enforces the launch order but no further dependencies, is that correct.
So a CUDA stream is more than a queue.
Is my understanding correct, or can someone point me to some resources or explain it, I would be grateful, thanks :)