asynchronous data tansfer b/w CPU and GPU

Accelerated Computing CUDA CUDA Programming and Performance
1

I know that : – Host memory loads and read-backs are synchronous and there is no need to put any sychronizing mechanism for it. The CPU after getting the control back from GPU ( after kernel launch) executes instructions till it encounters some GPU-CPU memory transfer. At that point it waits till the kernel finishes.

My question : 1. Suppose there are multiple kernels running then can the CPU read back the result of a previous kernel while the next kernel is running simultaneously (supposing there is no dependency in the previous kernel result and next kernel result)

  1. Is it not possible to fetch intermediate results? while the kernel is still running.

Thanks

2

I believe you can do this using Streams - look it up in the Programming Guide.

As far as I know, no.

3

Yes. You need to use the streams API and asynchronous memory transfers. See the programming guide for instructions and simpleStreams in the SDK for an example.

This is not possible.

4

Thanks a lot for the direction. I will check out the programming guide and come back with more questions. :)