Hello,
I am a beginner in the programming with cuda.
My Problem: I want synchronize my stream[n] with events in stream[n-2]. To make it more clearly, I have the follow Pseudo-/Fantasycode:
for streams[0 to 9] do{
memoryCopyToDevice();
computationOnGPU();
memoryCopytoHost();
}
Now I want synchronize the streams with follow two rules:
- In stream[n] the memoryCopytoDevice() begins when the computationOnGPU() is completed from the stream[n-2].
- In stream[n] the computationOnGPU begins when the memoryCopytoHost is completed from the stream[n-2].
Where I have to place the cudaEventRecords and the cudaEventSynchronize?
Best wishes from Germany
Tobias
PS.: Are cudaEventRecords and cudaEventSynchronize the right utilities to realize my synchronization-rules?
The correct API for this is cudaStreamWaitEvent()
Launch an event into stream X
insert in stream Y cudaStreamWaitEvent on event launched into stream X
when the CUDA processing in stream Y gets to the point of the cudaStreamWaitEvent, the stream processing in stream Y will halt until the event launched into stream X has completed.
Note that if you’re trying to create an ordinary pipelined algorithm, this is overly complex.
If you want an operation X to wait on an operation Y, the usual thing is to launch those into the same stream, Y before X. You might get sufficient throughput simply by re-using the same stream on for iteration n as you used for iteration n-2
cudaEventSynchronize() halts the CPU thread. That is probably not what you want.
If you google cudaStreamWaitEvent you’ll find examples of usage
Thank you :) I will try it
I dont know why, but it doesnt work with cudaStreamWaitEvent.
To make my wanted situation more clearly, I attach a picture".
Update: Something changed, maybe it works.
If you are working on a WDDM GPU in windows, this sort of detailed orchestration can get jumbled by the WDDM driver batching of commands.
I did it. Everything works for me now, as I wanted. Thanks a lot to Mr Crovella.
Best regards
Tobias
For what its worth, check out my video walkthrough on CUDA Synchronization:
https://cudaeducation.com/cudasynchronization/
-Cuda Education