# Local array is not removed from stack

**URL:** <https://forums.developer.nvidia.com/t/local-array-is-not-removed-from-stack/260346>\
**Category:** nvc, nvc++ and nvfortran\
**Created:** [July 19, 2023, 3:13pm UTC](https://forums.developer.nvidia.com/t/local-array-is-not-removed-from-stack/260346 "2023-07-19T15:13:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![christian.weiss](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/christian.weiss/32/327516_2.png) [@christian.weiss](https://forums.developer.nvidia.com/u/christian.weiss)\
**Post date:** [July 19, 2023, 3:13pm UTC](https://forums.developer.nvidia.com/t/local-array-is-not-removed-from-stack/260346/1 "2023-07-19T15:13:47Z")

</div>

Hi all,

in this code example, a local array `x2` is declared for each iteration and dummy data is copied into it:

```auto
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[]) {

  int nDim = 3;
  int nIter = 1000000;

  double x1[3];
  x1[0] = 1.0; x1[1] = 2.0; x1[2] = 3.0;

  #pragma omp target
  for(int i = 0; i < nIter; i++) {
      double x2[nDim]; // Segfault for large iteration counts (stacklimit exceeded?)
      for(int d = 0; d < nDim; ++d) {
        x2[d] = x1[d];
      }
  }
}

```

It is compiled with nvc(version 23.3), first ignoring OpenMP. To reproduce the error, set the stack limit to a fixed value, e.g. `ulimit -s 8192`. In this case, a segfault occurs when the binary is executed. The error can be circumvented by either setting a higher or unlimited ulimit, or by moving the declaration of `x2` out of the loop.  
Compiling with OpenMP offloading support, using `-mp=gpu -Minfo=all` gives another hint in the compilation diagnostics:

```auto
17, Accelerator restriction: unsupported statement type: opcode=DEALLOC

```

This statement refers to the declaration of `x2`, since it does not appear when this declaration is pulled out of the loop.

---

<div class="post-metadata">

**Author:** ![christian.weiss](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/christian.weiss/32/327516_2.png) [@christian.weiss](https://forums.developer.nvidia.com/u/christian.weiss)\
**Post date:** [July 20, 2023, 1:55pm UTC](https://forums.developer.nvidia.com/t/local-array-is-not-removed-from-stack/260346/2 "2023-07-20T13:55:06Z")

</div>

I just found out that the issue is resolved when `nDim` is declared as `const int`, in which case a static allocation is made. Apparently the compiler otherwise assumes that `nDim` can changes and makes a dynamic allocation. However, shouldn’t the memory be freed again in that case?

---

<div class="post-metadata">

**Author:** ![MatColgrove](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/matcolgrove/32/14047_2.png) [@MatColgrove](https://forums.developer.nvidia.com/u/MatColgrove)\
**Post date:** [July 24, 2023, 8:35pm UTC](https://forums.developer.nvidia.com/t/local-array-is-not-removed-from-stack/260346/3 "2023-07-24T20:35:36Z")

</div>

Hi Christian,

Apologies for the late response. This one took a bit of digging but since I was mentoring a GPU hackathon last week, I didn’t get a chance to investigate until today.

For the stack overflow issue on the host, we are putting the VLA on the stack but it doesn’t appear we’re using the correct stack save / stack restore, hence all allocation is getting on the stack. I’ve filed TPR #34033 and sent it to engineering for review.

For the unsupported DEALLOC statement, we do support free on the device, so it’s unclear what’s wrong. Hence I’ve filed TPR #34034 and will have engineering investigate.

However, I would highly recommend you not use VLAs in device code. This will cause each thread to allocate and presumably deallocate the memory which can be detrimental to performance as the allocates get serialized. Also, the default device heap is relatively small which can cause heap overflows. While the heap can be increased, it’s generally better to avoid device side allocation.

The better approach is to hoist the declaration of the VLA to before the loop and then add the array to a “private” clause. This will also work around the DEALLOC issue.

-Mat
