Errors when I use over 2GB of global mem, should have 4GB

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

I declare several arrays at the top of the module that contains my kernel, and when their total size exceeds about 2GB, I get this compile error:

ptxas /tmp/pgcudafor3hVbLGX00OKN.ptx, line 55; fatal : Parsing error near ‘-’: syntax error ptxas fatal : Ptx assembly aborted due to errors
PGF90-F-0000-Internal compiler error. pgnvd job exited with nonzero status code 0 (codefile.f: 830)

Line 830 is just the last line of routine that calls the kernel.

Any smaller array sizes work fine, and any larger ones give me the same error. I’m useing a Tesla C1060, and I should have 4GB of ram. When I’m not running the program and type pgaccelinfo, it lists “Global Memory Size 4294770688” and “Current free memory 4238348032”. When I make my arrays as large as possible (without causing the error - about 2GB) and do a pgaccelinfo, it says “Current free memory 2098568448”. So I should be able to make my arrays larger, right?

My block size isn’t changing. My grid dimensions increase as the arrays increase in size, but I haven’t gone over the grid size limits (65535x65535x1).
Any ideas?

Thanks!

2

Hi JDS7,

Try adding the flag “-Mlarge_arrays” is the arrays are dynamically allocated or -mcmodel=medium if they are static. Note that support for large arrays (>2GB) was added in the 11.9 release so you will need to update your compiler if it’s an earlier version.

Also, I think you need CUDA 4.0 as well, so for the 11.x compilers, add “-Mcuda=4.0”.

Hope this helps,
Mat