Event Synchronization

Hello,

I am a beginner in the programming with cuda.

My Problem: I want synchronize my stream[n] with events in stream[n-2]. To make it more clearly, I have the follow Pseudo-/Fantasycode:

for streams[0 to 9] do{
memoryCopyToDevice();
computationOnGPU();
memoryCopytoHost();
}

Now I want synchronize the streams with follow two rules:

  1. In stream[n] the memoryCopytoDevice() begins when the computationOnGPU() is completed from the stream[n-2].
  2. In stream[n] the computationOnGPU begins when the memoryCopytoHost is completed from the stream[n-2].

Where I have to place the cudaEventRecords and the cudaEventSynchronize?

Best wishes from Germany
Tobias

PS.: Are cudaEventRecords and cudaEventSynchronize the right utilities to realize my synchronization-rules?

The correct API for this is cudaStreamWaitEvent()

Launch an event into stream X

insert in stream Y cudaStreamWaitEvent on event launched into stream X

when the CUDA processing in stream Y gets to the point of the cudaStreamWaitEvent, the stream processing in stream Y will halt until the event launched into stream X has completed.

Note that if you’re trying to create an ordinary pipelined algorithm, this is overly complex.

If you want an operation X to wait on an operation Y, the usual thing is to launch those into the same stream, Y before X. You might get sufficient throughput simply by re-using the same stream on for iteration n as you used for iteration n-2

cudaEventSynchronize() halts the CPU thread. That is probably not what you want.

If you google cudaStreamWaitEvent you’ll find examples of usage

Thank you :) I will try it

I dont know why, but it doesnt work with cudaStreamWaitEvent.
To make my wanted situation more clearly, I attach a picture".

Update: Something changed, maybe it works.
EventSync.jpg

EventSync.jpg

If you are working on a WDDM GPU in windows, this sort of detailed orchestration can get jumbled by the WDDM driver batching of commands.

I did it. Everything works for me now, as I wanted. Thanks a lot to Mr Crovella.

Best regards
Tobias

For what its worth, check out my video walkthrough on CUDA Synchronization:

https://cudaeducation.com/cudasynchronization/

-Cuda Education
cudaeducation.com