Fortran async I/O issue

Accelerated Computing HPC Compilers Legacy PGI Compilers
#1

Hello,

I am using PGI CUDA Fortran compiler 19.10 on Power9. The program is compiled using IBM spectrum MPI PGI wrapper. I have problems using Fortran asynchronous I/O. Everything works perfectly fine up to around 2GB file size. After that I got an “FIO/stdio: Invalid argument” on a wait instruction. Deactivating the async behavior, I/O works correctly also for larger files (but the code is dramatically slow because it writes much data). Is it possible? How can I solve it?

Thanks,
Francesco

#2

We can try that here. Can you post a few snippets of how you are doing the i/o operation and the wait? We will see if we can reproduce it and send it to engineering. Might just be an improper int vs. size_t passed to an underlying system function. Or, we are using an interface that only takes an int.

#3

Hello,
this is a snippet of the described I/O part of the code. To activate async I/O, async is set equal to .true.
Basically, the first time the subroutine is called the file is opened, and then two writes are performed. The second write is asynchronous. At the next iteration, there is wait instruction and then write again. And so on. At the last iteration, there is a wait and a final close (if async=.false. the pattern is slightly different but it should not play a role here). When the files becomes larger than 2GB, I get the error.

subroutine wallp_io()
  use mod_var
  if(async) then
   if(icyc == ncyc0 + delta_cyc_p) then ! first write starts here
    open(122,file='WALLP/wallpressure_'//chx//'_'//chz//'.bin',form='unformatted',asynchronous="yes")
   else
    wait(122) 
   endif
  endif
!  
  wallpfield = wallpfield_gpu
!  
  if(async) then
   write(122, asynchronous="no") icyc,telaps,nx,nz !,i_skip_p,k_skip_p
   write(122, asynchronous="yes") wallpfield
  else
   open(122,file='WALLP/wallpressure_'//chx//'_'//chz//'.bin',form='unformatted',position="append")
   write(122) icyc,telaps,nx,nz !,i_skip_p,k_skip_p
   write(122) wallpfield
   close(122)
  endif
!
  if(async) then
   if(icyc == ncyc0+ncyc) then ! last write is completed here
    wait(122)
    close(122)
   endif
  endif
 endsubroutine wallp_io

Many thanks
Francesco

#4

I think I was able to reproduce your problem. I wrote a loop over your call, writing 64 Mbytes each time in a loop. I got this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
FIO/stdio: Invalid argument
FIO-F-/WAIT/unit=122/error code returned by host stdio - 22.
File name = ‘wallpressure.bin’, unformatted, sequential access async record = 64

#5

Yes, it seems the same error. I am now running using a work-around (basically creating new files when the size is too large) but it would be very useful to have it working using a single file. Let us know for any news.

Thanks,
Francesco

#6

We’ve identified the problem, and it will be fixed in an upcoming release. Sorry, I can’t think of any work-arounds other than what you have done.