unable to get the cpu and gpu to run in parallel

That is incorrect. It is very possible to overlap CPU and GPU computation within a single thread - all my linear algebra codes do this as a basic design tenet. Cuda has been fully asynchronous since 1.0 was released three years ago. I am not sure what you are doing wrong (and whether this is actually an instrumentation/measurement problem), but be assured that you are doing something wrong.

Yay! Thank you! That’s what I was looking for.

Yay! Thank you! That’s what I was looking for.

Maybe you can try to use CUDA Streams and AsyncMemCopies ? I think they are ment for that exactly. Am I worng?

Maybe you can try to use CUDA Streams and AsyncMemCopies ? I think they are ment for that exactly. Am I worng?

You need to set the “device options” using some CUDA API call… YOu can control the behaviour of “cudaThreadSynchronize” using that call…
I fail to remember the API though… cudaSetDeviceFlags() or cudaSetDeviceOptions()??? something… check the manual.

OR

You can run another desktop close to your GPU machine… The CPU on the other machine and GPU in your machine would run in parallel…
Hayyoo… Hayyoo…

You need to set the “device options” using some CUDA API call… YOu can control the behaviour of “cudaThreadSynchronize” using that call…
I fail to remember the API though… cudaSetDeviceFlags() or cudaSetDeviceOptions()??? something… check the manual.

OR

You can run another desktop close to your GPU machine… The CPU on the other machine and GPU in your machine would run in parallel…
Hayyoo… Hayyoo…

Hi, thanks for the suggestion, but those options control how cudaThreadSynchronize() waits. In my case it shouldn’t be waiting for anything at all.

I think that was a joke… I hope that was a joke. :)

Hi, thanks for the suggestion, but those options control how cudaThreadSynchronize() waits. In my case it shouldn’t be waiting for anything at all.

I think that was a joke… I hope that was a joke. :)

Yeah, it was a joke. :-)

Yeah, it was a joke. :-)

Have you tried running it with the NSight development addon? It would be interesting to see how this showed up there

Have you tried running it with the NSight development addon? It would be interesting to see how this showed up there