# Parallel STL algorithms may run sequentially with thrust::counting\_iterator

**URL:** <https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610>\
**Category:** nvc, nvc++ and nvfortran\
**Created:** [September 26, 2023, 8:24am UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610 "2023-09-26T08:24:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![christos.akrivopoulos](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@christos.akrivopoulos](https://forums.developer.nvidia.com/u/christos.akrivopoulos)\
**Post date:** [September 26, 2023, 8:24am UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/1 "2023-09-26T08:24:41Z")

</div>

Hi,

I’m using [HPC SDK 23.7](https://developer.nvidia.com/nvidia-hpc-sdk-237-downloads) and compiling without `-stdpar=multicore`, but with `-DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_TBB`.

I’ve noticed that passing `thrust::counting_iterator`s to parallel STL algorithms such as `for_each`, `transform`, `inclusive_scan`, and `transform_inclusive_scan` results in the algorithms running sequentially.

Example:

```cpp
std::for_each(
  std::execution::par,
  thrust::make_counting_iterator(0),
  thrust::make_counting_iterator(1000000),
  [](int i) { /* do something */ }
);

```

The same algorithms run in parallel when using my own or oneAPI’s counting iterator implementation (see [here](https://oneapi-src.github.io/oneDPL/parallel_api/iterators.html)). They also run in parallel when using `-stdpar=multicore` (except for scan algorithms).

NOTE: Thrust’s counting iterators are random access iterators, which is in agreement with [this](https://docs.nvidia.com/hpc-sdk/compilers/c++-parallel-algorithms/index.html#iterators).

Any idea why this is so?

Regards,  
Christos

---

<div class="post-metadata">

**Author:** ![MatColgrove](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/matcolgrove/32/14047_2.png) [@MatColgrove](https://forums.developer.nvidia.com/u/MatColgrove)\
**Post date:** [September 26, 2023, 4:07pm UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/2 "2023-09-26T16:07:43Z")

</div>

Hi Christos,

> [@christos.akrivopoulos](#):
>
> Any idea why this is so?

I haven’t used Thrust counting iterators before but ported one of my codes, LULESH, to use it and it runs in parallel when targeting multicore. Not sure why it’s not for you.

Can you post a minimal reproducing example so I can investigate?

Thanks,  
Mat

---

<div class="post-metadata">

**Author:** ![christos.akrivopoulos](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@christos.akrivopoulos](https://forums.developer.nvidia.com/u/christos.akrivopoulos)\
**Post date:** [September 27, 2023, 9:07am UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/3 "2023-09-27T09:07:26Z")

</div>

Hi Mat,

Thanks for your reply.

Apologies, my description is not accurate. The problem happens without `-stdpar=multicore` and with `-DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_TBB`.

Can we delete this topic? I’ll study the matter more deeply and come back with a new topic.

Thanks,  
Christos

---

<div class="post-metadata">

**Author:** ![MatColgrove](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/matcolgrove/32/14047_2.png) [@MatColgrove](https://forums.developer.nvidia.com/u/MatColgrove)\
**Post date:** [September 27, 2023, 5:17pm UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/4 "2023-09-27T17:17:00Z")

</div>

No problem.

> Can we delete this topic?

There’s a trash can icon you can use to delete topics, but I believe this just flags it and an admin needs to do the the actual deletion.

Though, there’s no issue with keep it.

---

<div class="post-metadata">

**Author:** ![christos.akrivopoulos](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@christos.akrivopoulos](https://forums.developer.nvidia.com/u/christos.akrivopoulos)\
**Post date:** [September 28, 2023, 8:28am UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/5 "2023-09-28T08:28:22Z")

</div>

Hi Mat,

Thanks.

I’ve updated the title and description of the topic.

In brief, here’s the problem:

If I use `-DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_TBB` (or if I choose TBB as Thrust’s host system in CMake), without `-stdpar=multicore`, passing `thrust::counting_iterator` will result in algorithms running serially.

I am attaching a small program where you can see that. It performs a math operation (vector norm) on several vectors: with and without counting iterators. On my PCs, the version using counting iterators is slower, when I compile it like this:

```bash
nvc++ -O3 -fast -DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_TBB -I/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/cuda/include/ test.cpp -o test -ltbb

```

[test.txt](https://forums.developer.nvidia.com/uploads/short-url/pN18KABCbijTjsIhQ8nlgDDa9OV.txt) (2.3 KB)

Thanks,  
Christos

---

<div class="post-metadata">

**Author:** ![MatColgrove](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/matcolgrove/32/14047_2.png) [@MatColgrove](https://forums.developer.nvidia.com/u/MatColgrove)\
**Post date:** [September 28, 2023, 5:50pm UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/6 "2023-09-28T17:50:57Z")

</div>

Hi Christos,

Since this sounds like an issue with Thrust or possibly TBB, you should consider reporting this on the Thrust GitHub:  
[Issues · NVIDIA/thrust · GitHub](https://github.com/NVIDIA/thrust/issues)

-Mat

---

<div class="post-metadata">

**Author:** ![christos.akrivopoulos](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@christos.akrivopoulos](https://forums.developer.nvidia.com/u/christos.akrivopoulos)\
**Post date:** [September 29, 2023, 7:36am UTC](https://forums.developer.nvidia.com/t/parallel-stl-algorithms-may-run-sequentially-with-thrust-counting-iterator/267610/7 "2023-09-29T07:36:40Z")

</div>

Hi Mat,

That’s a good idea, thanks.

However, it might be worth noticing that directly calling `thrust::transform` in the example program in my previous post works as expected with counting iterators and TBB (i.e. the algorithm runs in parallel).

Cheers,  
Christos