Managed Memory

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

Hi,

This is my code :

ATTRIBUTES(global) SUBROUTINE collapse3Dto2D ( &
		kpt, ipointerStart, &
		igall_size, igall, & 
		csm_row, csm_col, csm, & 
		cx_size, cx, &
		cvs1_row, cvs1_col, cvs1_layer, cvs1 )
	USE cudafor
	IMPLICIT NONE
	INTEGER :: kpt, ipointerStart
	INTEGER :: igall_size, csm_row, csm_col, cx_size, cvs1_row, cvs1_col, cvs1_layer
	INTEGER :: igall(igall_size)
	COMPLEX*16 :: csm(csm_row, csm_col), cx(cx_size), cvs1(cvs1_row, cvs1_col, cvs1_layer)
	
	!Working variable
	INTEGER :: ipointer, ns, ipindexStart, ipindexEnd, ipindex
	
	ipointer = ipointerStart - 1 + (blockidx%x - 1) * blockdim%x + threadidx%x
	ns = (ipointer - 1) * kpt
	ipindexStart = igall(ipointer)
	ipindexEnd = igall(ipointer + 1) - 1
	DO ipindex = ipindexStart, ipindexEnd
		csm(1:2, ns+1:ns+kpt) = csm(1:2, ns+1:ns+kpt) + cvs1(1:2, 1:kpt, ipindex) * cx(ipindex) !<--- PROBLEM
	ENDDO
	print *, "Done update"
END SUBROUTINE collapse3Dto2D

I compiled with this: pgfortran -mp -Mcuda=cc60 -Mlarge_arrays .f

Lastly, I get this: 0: copyin Memcpy (dev=0x0x7f0732401a00, host=0x0x7ffc96b0a7cc, size=4) FAILED: 77(an illegal memory access was encountered)

I narrowed down that the problem lies in this line:

csm(1:2, ns+1:ns+kpt) = csm(1:2, ns+1:ns+kpt) + cvs1(1:2, 1:kpt, ipindex) * cx(ipindex)

All variables and array are declared as managed in the host part.

Any explanation ?

Thanks,
Phoon

2

Hi Phoon,

An “illegal memory access” error is a generic error, similar to a segmentation violation on the host, where the device is accessing a bad address. There could be many causes to this error.

Things to look for:

  1. Is the program writing or reading beyond the end of any of the arrays?
  2. Are the indices being passed in correct? I see that you’re not using the “value” attribute on the integer arguments. In this case then you need to make sure that the integers are device variables and properly initialized, Managed is only available for dynamic data (i.e. allocatable arrays/scalars) so can’t be used with non-allocated scalars.
  3. Are all the arrays either device or managed?

If you can provide a full reproducing example (either posted here or sent to PGI Customer Service, “support@pgroup.com”), I’d be happy to take a look and see if I can help file the specific failure.

-Mat

3

Hi Mat,

Thank you for your reply. To answer your question,

  1. Is the program writing or reading beyond the end of any of the arrays?

I have tested the program with CPU version and does not have any segmentation fault. So the direct translation from Host code to Cuda Fortran shouldn’t be a problem.

  1. Are the indices being passed in correct? I see that you’re not using the “value” attribute on the integer arguments. In this case then you need to make sure that the integers are device variables and properly initialized, Managed is only available for dynamic data (i.e. allocatable arrays/scalars) so can’t be used with non-allocated scalars.

I am not aware that scalar value do not have managed attributes. I tried to compile the scalar variable with “managed” attribute and it goes through, so I presumed that it’s a valid move. To clarify, for :
INTEGER, MANAGED :: kpt

I should change to:
INTEGER, VALUE :: kpt?

  1. Are all the arrays either device or managed?

All allocatable arrays are declared as managed

4

I have tested the program with CPU version and does not have any segmentation fault. So the direct translation from Host code to Cuda Fortran shouldn’t be a problem.

The GPU is more sensitive to out-of-bounds errors. Just because the program doesn’t crash on the CPU does not mean that the code is not accessing the array out of bounds. I’m not saying that this is the problem here, just that you can’t assume that since it works on the CPU, it’s not a potential issue.

I am not aware that scalar value do not have managed attributes. I tried to compile the scalar variable with “managed” attribute and it goes through, so I presumed that it’s a valid move. To clarify, for :
INTEGER, MANAGED :: kpt

I should change to:
INTEGER, VALUE :: kpt?

No, manged is only available for dynamic data (or fixed size arrays). This is probably the issue. Try removing managed from the scalar declaration and add value attribute to the argument declaration in “collaspe3Dto3D”.

-Mat