Best way to synchronise two stream from different gpus

[guilhermehartmann]

Hi,

I am currently running some work in multi-gpu what I would like to synchronise directly on the gpu. As I understand the events and streams are per gpu, so I we need to device synchronise on the host before issuing commands. Is that correct ?

Please let me know if you have any best practices.

Thank you,
Guilherme

[striker159]

Dependencies between streams of different gpus are no different than those betweens streams on the same gpu.

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#stream-and-event-behavior

cudaStreamWaitEvent() will succeed even if the input stream and input event are associated to different devices. cudaStreamWaitEvent() can therefore be used to synchronize multiple devices with each other.

[guilhermehartmann]

ha, Thank you !