# Synchronization between Kernel calls

**URL:** <https://forums.developer.nvidia.com/t/synchronization-between-kernel-calls/23336>\
**Category:** CUDA Programming and Performance\
**Created:** [July 3, 2011, 11:35am UTC](https://forums.developer.nvidia.com/t/synchronization-between-kernel-calls/23336 "2011-07-03T11:35:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gabrielmagno](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/gabrielmagno/32/12173_2.png) [@gabrielmagno](https://forums.developer.nvidia.com/u/gabrielmagno)\
**Post date:** [July 3, 2011, 11:35am UTC](https://forums.developer.nvidia.com/t/synchronization-between-kernel-calls/23336/1 "2011-07-03T11:35:08Z")

</div>

Is the invocation of a kernel synchronous?

For example, I have this program:

```auto
kernel_1 <<<...>>> (...);

for (i = 0; i < k; i++) {

   kernel_2 <<<...>>> (...);

   kernel_3 <<<...>>> (...);

}

```

Each kernel call must be done just when all the previous kernel’s threads are done.

Some threads may take more time than others.

I tried using “cudaDeviceSynchronize()” after all the kernel calls, but it didn’t work.

I’m, getting wrong random results in my calculation.

---

<div class="post-metadata">

**Author:** ![tera](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tera](https://forums.developer.nvidia.com/u/tera)\
**Post date:** [July 3, 2011, 12:00pm UTC](https://forums.developer.nvidia.com/t/synchronization-between-kernel-calls/23336/2 "2011-07-03T12:00:40Z")

</div>

Kernels in the same stream are invoked sequentially, so you don’t need any extra synchronization in this example. If your kernel returns unexpected results, it must be for some other reason.

---

<div class="post-metadata">

**Author:** ![akavo](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@akavo](https://forums.developer.nvidia.com/u/akavo)\
**Post date:** [July 4, 2011, 1:32am UTC](https://forums.developer.nvidia.com/t/synchronization-between-kernel-calls/23336/3 "2011-07-04T01:32:56Z")

</div>

> [@](#):
>
> Is the invocation of a kernel synchronous?
> 
> For example, I have this program:
> 
> ```auto
> kernel_1 <<<...>>> (...);
> 
> for (i = 0; i < k; i++) {
> 
> kernel_2 <<<...>>> (...);
> 
> kernel_3 <<<...>>> (...);
> 
> }
> 
> ```
> 
> Each kernel call must be done just when all the previous kernel’s threads are done.
> 
> Some threads may take more time than others.
> 
> I tried using “cudaDeviceSynchronize()” after all the kernel calls, but it didn’t work.
> 
> I’m, getting wrong random results in my calculation.

While kernel calls are asynchronous, they are so with respect to the CPU. As tera said, threads in the same stream execute in order in FIFO fashion.