I want to synchronize CUDA streams

Accelerated Computing CUDA CUDA Programming and Performance
1

I am currently performing asynchronous operations using five streams in CUDA.
I want to complete the code in a specific order: Memcpy - cudaMemcpy (Host to Device) - Kernel - cudaMemcpy (Device to Host) - Memcpy. I am encountering an issue where the cudaMemcpy (Device to Host) needs to be completed before I can access the destination address in the last Memcpy. I tried using stream synchronization, but encountered a problem where it breaks after one cycle.
Is there a way to stop each stream while maintaining asynchronous behavior, ensuring that cudaMemcpy (Device to Host) is completed before accessing Memcpy on the CPU?

The following is the result of monitoring the code with errors using Nsight.

image
image864×129 8 KB

2

The general stream method to ensure that operation B does not begin until operation A is complete is to launch them into the same stream.

You can also look at cudaStreamWaitEvent() as another option.

3
5

I’ve tried almost everything there, but the issue persists. Asynchronous execution seems to work, but at the point of Stream synchronization, it appears as if the GPU is waiting for all Streams to complete. Should each Stream exist in a different thread.
I’m really curious.

6

I just resolved it using a callback function and cudaLaunchHostFunc ! Thank you very much