Is there anyway to query the number of remaining operations in a stream?

Accelerated Computing CUDA CUDA Programming and Performance
1

After creating a stream, we can launch operations like cudaMemcpyAsync/kernels to it. I want to know how many operations are there in the stream at any time. Does there exist any cudart api or driver api to do it? If CDP is used, does similar api exist? Thanks!

2

nothing is provided by CUDA to do that.

You could use events and query the events to find out where stream processing has reached.

You can of course also use cudaStreamQuery(). That’s a binary finished/not finished indication.

3

Thanks! I’ll try that.

4

Also cudaStreamAddCallback with a callback just increasing a counter (atomic or not?) could be a solution.

2 Likes