Library Linkage problems

[jmckennon]

Hi,

I’m trying to compile my code and I appear to have many linkage errors. It looks like I’m missing linkage to some sort of libraries. I’m not particularly well versed with pgf so I’m not sure which libraries I’m missing linkage to. Can anyone point me in the right direction?

/home/gravity/FLASH3.2/object/./rieman.cuf:188: undefined reference to `pgf90_dev_alloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:188: undefined reference to `pgf90_dev_alloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:188: undefined reference to `pgf90_dev_alloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:188: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `pgf90_dev_copyin'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `pgf90_dev_configure_call'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `pgf90_dev_mkdesc'
/home/gravity/FLASH3.2/object/./rieman.cuf:193: undefined reference to `pgf90_dev_allocated'
/home/gravity/FLASH3.2/object/./rieman.cuf:203: undefined reference to `pgf90_dev_dealloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:203: undefined reference to `pgf90_dev_allocated'
/home/gravity/FLASH3.2/object/./rieman.cuf:203: undefined reference to `pgf90_dev_dealloc'
/home/gravity/FLASH3.2/object/./rieman.cuf:203: undefined reference to `pgf90_dev_allocated'
/home/gravity/FLASH3.2/object/./rieman.cuf:203: undefined reference to `pgf90_dev_dealloc'
rieman.o: In function `riemancalc_rieman_kernel___entry':
/home/gravity/FLASH3.2/object/./rieman.cuf:208: undefined reference to `cudaSetupArgument'
/home/gravity/FLASH3.2/object/./rieman.cuf:208: undefined reference to `cudaSetupArgument'
/home/gravity/FLASH3.2/object/./rieman.cuf:208: undefined reference to `cudaLaunch'
rieman.o: In function `..cuda_fortran_constructor_1':
<unknown>:63: undefined reference to `__cudaRegisterFatBinary'
<unknown>:63: undefined reference to `__cudaRegisterFunction'
<unknown>:63: undefined reference to `__cudaUnregisterFatBinary'
rieman.o: In function `.STATICS4':
rieman.cuf:(.data+0xc14): undefined reference to `cudafor_'
rieman.cuf:(.data+0xc1c): undefined reference to `iso_c_binding_'
rieman.cuf:(.data+0xc28): undefined reference to `pgf90_compiled'
Burn.o:(.data+0x0): undefined reference to `pgf90_compiled'
Burn_computeDt.o:(.data+0x0): undefined reference to `pgf90_compiled'
Burn_finalize.o:(.data+0x0): undefined reference to `pgf90_compiled'
Burn_init.o:(.data+0x0): undefined reference to `pgf90_compiled'
Burn_nseAtDens.o:Burn_nseAtDens.F90:(.data+0x10): more undefined references to `pgf90_compiled' follow
Driver_abortFlash.o: In function `driver_abortflash_':
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:53: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:53: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:53: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:53: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:55: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:55: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:55: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:55: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:56: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:56: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:56: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:56: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:58: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:58: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:58: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:58: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:58: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:61: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:61: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:61: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:61: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:62: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:62: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:62: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:62: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:65: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:65: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:65: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:65: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:67: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:67: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:67: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:67: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:67: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90_lentrim'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90io_ldw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:69: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:84: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_abortFlash.F90:104: undefined reference to `pgf90_stop'
Driver_abortFlash.o: In function `.STATICS1':
Driver_abortFlash.F90:(.data+0x248): undefined reference to `pgf90_compiled'
Driver_checkMPIErrorCode.o: In function `driver_checkmpierrorcode_':
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:92: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:95: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:99: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:109: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:109: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:110: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:113: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:113: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:113: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:113: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:121: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:123: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:126: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:126: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:126: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:126: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:134: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:134: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:135: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:138: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:138: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:138: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_checkMPIErrorCode.F90:138: undefined reference to `pgf90io_ldw_end'
Driver_checkMPIErrorCode.o: In function `.C1_283':
Driver_checkMPIErrorCode.F90:(.data+0x1d0): undefined reference to `pgf90_compiled'
Driver_computeDt.o: In function `driver_computedt_':
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:432: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:468: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:469: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:470: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:470: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pghpfio_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:476: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pghpfio_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_computeDt.F90:477: undefined reference to `pgf90io_fmtw_end'
Driver_computeDt.o: In function `.STATICS1':
Driver_computeDt.F90:(.data+0x898): undefined reference to `pgf90_compiled'
Driver_evolveFlash.o: In function `driver_evolveflash_':
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_sc_i_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90_adjustl'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90_adjustl'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:99: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90_adjustl'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:109: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:113: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90_adjustl'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_sc_f_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_intern_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90_adjustl'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90_trim'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmt_write'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:115: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:275: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:275: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:275: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:275: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:283: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:283: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:283: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:283: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:290: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:290: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:290: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_evolveFlash.F90:290: undefined reference to `pgf90io_ldw_end'
Driver_evolveFlash.o: In function `.C1_283':
Driver_evolveFlash.F90:(.data+0x238): undefined reference to `pgf90_compiled'
Driver_finalizeFlash.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_finalizeSourceTerms.o:(.data+0x0): undefined reference to `pgf90_compiled'
Driver_getDt.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_getElapsedWCTime.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_getNStep.o:(.data+0x8): more undefined references to `pgf90_compiled' follow
Driver_getTimeStamp.o: In function `driver_gettimestamp_':
/home/gravity/FLASH3.2/object/./Driver_getTimeStamp.F90:37: undefined reference to `pgf90_str_copy'
Driver_getTimeStamp.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_init.o: In function `.C1_287':
Driver_init.F90:(.data+0x120): undefined reference to `pgf90_compiled'
Driver_initFlash.o: In function `driver_initflash_':
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:121: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:121: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:121: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:121: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:134: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:134: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:134: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:134: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:142: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:142: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:142: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:142: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:150: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:150: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:150: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:150: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:165: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:165: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:165: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:165: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:170: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:170: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:170: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:170: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:175: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:175: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:175: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:175: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:177: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:177: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:177: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:177: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:181: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:181: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:181: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:181: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:185: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:185: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:185: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:185: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:198: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:198: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:198: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:198: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:200: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:200: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:200: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_initFlash.F90:200: undefined reference to `pgf90io_ldw_end'
Driver_initFlash.o: In function `.C1_283':
Driver_initFlash.F90:(.data+0x210): undefined reference to `pgf90_compiled'
Driver_initMaterialProperties.o:(.data+0x0): undefined reference to `pgf90_compiled'
Driver_initParallel.o: In function `.C1_283':
Driver_initParallel.F90:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_initSourceTerms.o:(.data+0x0): undefined reference to `pgf90_compiled'
Driver_putTimeStamp.o: In function `driver_puttimestamp_':
/home/gravity/FLASH3.2/object/./Driver_putTimeStamp.F90:37: undefined reference to `pgf90_str_copy'
Driver_putTimeStamp.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_sendOutputData.o: In function `.C1_344':
Driver_sendOutputData.F90:(.data+0x38): undefined reference to `pgf90_compiled'
Driver_sourceTerms.o:(.data+0x8): undefined reference to `pgf90_compiled'
Driver_verifyInitDt.o: In function `driver_verifyinitdt_':
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_sc_f_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_sc_f_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Driver_verifyInitDt.F90:169: undefined reference to `pgf90io_ldw_end'
Driver_verifyInitDt.o: In function `.STATICS1':
Driver_verifyInitDt.F90:(.data+0x170): undefined reference to `pgf90_compiled'
Eos.o: In function `eos_':
/home/gravity/FLASH3.2/object/./Eos.F90:269: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:269: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:269: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:269: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:271: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:271: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:271: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:273: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:273: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:280: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:282: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:282: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:282: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:284: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:284: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:284: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:286: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:286: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:293: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:295: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:295: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:295: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:297: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:297: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:297: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:299: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:299: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:311: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:312: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:312: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:315: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:316: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:316: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:319: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:320: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:320: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:324: undefined reference to `__c_mzero4'
/home/gravity/FLASH3.2/object/./Eos.F90:330: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:333: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:333: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:342: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:345: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:345: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:356: undefined reference to `__c_mzero4'
/home/gravity/FLASH3.2/object/./Eos.F90:360: undefined reference to `__c_mzero4'
/home/gravity/FLASH3.2/object/./Eos.F90:364: undefined reference to `__c_mzero4'
/home/gravity/FLASH3.2/object/./Eos.F90:369: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:370: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:370: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:370: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos.F90:378: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos.F90:379: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Eos.F90:379: undefined reference to `pgf90_dealloc'
Eos.o: In function `.C1_349':
Eos.F90:(.data+0x150): undefined reference to `pgf90_compiled'
Eos_finalize.o:(.data+0x0): undefined reference to `pgf90_compiled'
Eos_getData.o: In function `.C1_288':
Eos_getData.F90:(.data+0x8): undefined reference to `pgf90_compiled'
Eos_getParameters.o:(.data+0x0): undefined reference to `pgf90_compiled'
Eos_guardCells.o: In function `eos_guardcells_':
/home/gravity/FLASH3.2/object/./Eos_guardCells.F90:60: undefined reference to `pgf90_mcopy4'
Eos_guardCells.o:(.data+0x0): undefined reference to `pgf90_compiled'
Eos_init.o: In function `.C1_288':
Eos_init.F90:(.data+0x88): undefined reference to `pgf90_compiled'
Eos_putData.o: In function `eos_putdata_':
/home/gravity/FLASH3.2/object/./Eos_putData.F90:141: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:155: undefined reference to `__c_mzero4'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_a'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:164: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:166: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:168: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:176: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:229: undefined reference to `pgf90_allocated'
/home/gravity/FLASH3.2/object/./Eos_putData.F90:232: undefined reference to `pgf90_dealloc'
Eos_putData.o: In function `.C1_288':
Eos_putData.F90:(.data+0x248): undefined reference to `pgf90_compiled'
Eos_wrapped.o: In function `.C1_288':
Eos_wrapped.F90:(.data+0x8): undefined reference to `pgf90_compiled'
Flash.o: In function `flash':
/home/gravity/FLASH3.2/object/./Flash.F90:38: undefined reference to `pghpf_init'
Flash.o: In function `.C1_283':
Flash.F90:(.data+0x8): undefined reference to `pgf90_compiled'
Gravity_accelAtCoords.o: In function `gravity_accelatcoords_':
/home/gravity/FLASH3.2/object/./Gravity_accelAtCoords.F90:57: undefined reference to `__c_mzero4'
Gravity_accelAtCoords.o:(.data+0x0): undefined reference to `pgf90_compiled'
Gravity_accelListOfBlocks.o:(.data+0x0): undefined reference to `pgf90_compiled'
Gravity_accelOneRow.o:(.data+0x0): undefined reference to `pgf90_compiled'
Gravity_computeDt.o: In function `.C1_301':
Gravity_computeDt.F90:(.data+0x8): undefined reference to `pgf90_compiled'
Gravity_finalize.o:(.data+0x0): undefined reference to `pgf90_compiled'
Gravity_init.o:Gravity_init.F90:(.data+0x90): more undefined references to `pgf90_compiled' follow
Grid_dump.o: In function `grid_dump_':
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pghpfio_fmt_write'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:78: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:80: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:80: undefined reference to `pgf90io_encode_fmt'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:80: undefined reference to `pgf90io_fmtw_init'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:80: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:81: undefined reference to `pghpfio_fmt_write'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:81: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:81: undefined reference to `pgf90io_fmtw_end'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:82: undefined reference to `pgf90_str_copy'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:117: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:117: undefined reference to `pgf90io_open'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:117: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_dump.F90:117: undefined reference to `pgf90io_close'
Grid_dump.o: In function `.STATICS1':
Grid_dump.F90:(.data+0x78): undefined reference to `pgf90_compiled'
Grid_fillGuardCells.o: In function `grid_fillguardcells_':
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:299: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:299: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:299: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:299: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:336: undefined reference to `pgf90_mzero4'
/home/gravity/FLASH3.2/object/./Grid_fillGuardCells.F90:418: undefined reference to `pgf90_mset4'
Grid_fillGuardCells.o: In function `.STATICS1':
Grid_fillGuardCells.F90:(.data+0x160): undefined reference to `pgf90_compiled'
Grid_finalize.o: In function `grid_finalize_':
/home/gravity/FLASH3.2/object/./Grid_finalize.F90:27: undefined reference to `pgf90_dealloc'
/home/gravity/FLASH3.2/object/./Grid_finalize.F90:27: undefined reference to `pgf90_dealloc'
Grid_finalize.o:(.data+0x8): undefined reference to `pgf90_compiled'
Grid_getBlkBC.o: In function `grid_getblkbc_':
/home/gravity/FLASH3.2/object/./Grid_getBlkBC.F90:75: undefined reference to `pgf90_mset4'
/home/gravity/FLASH3.2/object/./Grid_getBlkBC.F90:95: undefined reference to `pgf90_mcopy4'
Grid_getBlkBC.o: In function `.C1_285':
Grid_getBlkBC.F90:(.data+0x18): undefined reference to `pgf90_compiled'
Grid_getBlkBoundBox.o: In function `grid_getblkboundbox_':
/home/gravity/FLASH3.2/object/./Grid_getBlkBoundBox.F90:76: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_getBlkBoundBox.F90:76: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_getBlkBoundBox.F90:76: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkBoundBox.F90:76: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkBoundBox.F90:76: undefined reference to `pgf90io_ldw_end'
Grid_getBlkBoundBox.o: In function `.C1_283':
Grid_getBlkBoundBox.F90:(.data+0x88): undefined reference to `pgf90_compiled'
Grid_getBlkCenterCoords.o:(.data+0x0): undefined reference to `pgf90_compiled'
Grid_getBlkCornerID.o: In function `grid_getblkcornerid_':
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_end'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_sc_i_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pghpfio_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkCornerID.F90:122: undefined reference to `pgf90io_ldw_end'
Grid_getBlkCornerID.o: In function `.C1_576':
Grid_getBlkCornerID.F90:(.data+0xe8): undefined reference to `pgf90_compiled'
Grid_getBlkData.o:(.data+0x8): undefined reference to `pgf90_compiled'
Grid_getBlkIndexLimits.o: In function `grid_getblkindexlimits_':
/home/gravity/FLASH3.2/object/./Grid_getBlkIndexLimits.F90:116: undefined reference to `pgf90_mset4'
Grid_getBlkIndexLimits.o: In function `.C1_285':
Grid_getBlkIndexLimits.F90:(.data+0x38): undefined reference to `pgf90_compiled'
Grid_getBlkPhysicalSize.o:(.data+0x0): undefined reference to `pgf90_compiled'
Grid_getBlkPtr.o: In function `grid_getblkptr_':
/home/gravity/FLASH3.2/object/./Grid_getBlkPtr.F90:218: undefined reference to `pgf90io_src_info'
/home/gravity/FLASH3.2/object/./Grid_getBlkPtr.F90:218: undefined reference to `pgf90io_ldw_init'
/home/gravity/FLASH3.2/object/./Grid_getBlkPtr.F90:218: undefined reference to `pgf90io_ldw'
/home/gravity/FLASH3.2/object/./Grid_getBlkPtr.F90:218: undefined reference to `pgf90io_ldw_end'
Grid_getBlkPtr.o: In function `.C1_283':
Grid_getBlkPtr.F90:(.data+0xc0): undefined reference to `pgf90_compiled'
Grid_getBlkRefineLevel.o:(.data+0x0): undefined reference to `pgf90_compiled'
Grid_getBlkType.o:(.data+0x0): undefined reference to `pgf90_compiled'
Grid_getCellCoords.o:(.data+0x8): undefined reference to `pgf90_compiled'
Grid_getDeltas.o:(.data+0x8): undefined reference to `pgf90_compiled'
Grid_getFluxData.o:Grid_getFluxData.F90:(.data+0x10): more undefined references to `pgf90_compiled' follow
Grid_getPlaneData.o: In function `grid_getplanedata_':
/home/gravity/FLASH3.2/object/./Grid_getPlaneData.F90:529: undefined reference to `pgf90_alloc'
/home/gravity/FLASH3.2/object/./Grid_getPlaneData.F90:531: undefined reference to `__c_mcopy4'
/home/gravity/FLASH3.2/object/./Grid_getPlaneData.F90:532: undefined reference to `__c_mcopy4'
/home/gravi

[MatColgrove]

Hi jmckennon,

How are you linking your program? These errors mean that you’re not linking in the PGI runtime libraries. So you’re most likely using ld directly or another vendor’s compiler to link.

The easiest thing to do would be to link using a PGI compiler and the “-Mcuda” flag then the correct libraries will be added for you. Otherwise, you’ll need to add the PGI runtime libraries to your link line (i.e. “-rpath /usr/pgi/linux86-64/2010/cuda/2.3/lib -lcudafor -L/usr/pgi/linux86-64/2010/cuda/2.3/lib -lcudart -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc” be sure to change the -L path to match your system).

Hope this helps,
Mat

[jmckennon]

Hi Mat,
I run my program by doing gmake --jobs=8. The code for my Makefile is below. Where would I include the pgi runtime libraries in this file? I really appreciate the help!

#       Makefile.h file for alc.llnl.gov
#
#	FLASH makefile definitions for Linux (Intel compiler)


#----------------------------------------------------------------------------
# Set the HDF/HDF5 library paths -- these need to be updated for your system
#----------------------------------------------------------------------------


HDF5path   = /home/gravity/hdf5-1.6.10/hdf5
MPIpath    = /home/gravity/openmpi-1.4.2/ompi/include

PAPI_PATH  = 
PAPI_FLAGS = 

ZLIB_PATH  =

NCMPI_PATH = /usr/local/pnetcdf-1.0.0/gcc
MPE_PATH   =

#----------------------------------------------------------------------------
# Compiler and linker commands
#
#  We use the f90 compiler as the linker, so some C libraries may explicitly
#  need to be added into the link line.
#----------------------------------------------------------------------------

FCOMP      = mpif90
CCOMP      = mpicc
CPPCOMP    = mpicxx
LINK       = mpif90


# pre-processor flag


PP         = -D

#-----------------------------------------------------------------------------
# Compilation flags
#
#  Three sets of compilation/linking flags are defined: one for optimized code
#  code ("-opt"), one for debugging ("-debug"), and one for testing ("-test").
#  Passing these flags to the setup script will cause the value associated with
#  the corresponding keys (i.e. those ending in "_OPT", "_DEBUG", or "_TEST") to
#  be incorporated into the final Makefile. For example, passing "-opt" to the
#  setup script will cause the flags following "FFLAGS_OPT" to be assigned to
#  "FFLAGS" in the final Makefile. If none of these flags are passed, the default
#  behavior will match that of the "-opt" flag.
#  In general, "-opt" is meant to optimize compilation and linking. "-debug"
#  should enable runtime bounds checking, debugger symbols, and other compiler-
#  specific debugging options. "-test" is useful for testing different
#  combinations of compiler flags particular to your individual system.
#----------------------------------------------------------------------------
FFLAGS_OPT = -c -O2
FFLAGS_DEBUG = -g -c
FFLAGS_TEST = -c

CFLAGS_OPT = -O2  -c
CFLAGS_DEBUG = -g -c
CFLAGS_TEST = -c

FFLAGS_PAPI  = -I$(PAPI_PATH)/include

F90FLAGS     =

#CFLAGS       = -c -O3 -tpp7 -march=pentium4 -mcpu=pentium4 -ip -unroll \
#               -D_LARGEFILE64_SOURCE

CFLAGS_OPT =   -I$(MPIpath)/include -c -O2 -D_LARGEFILE64_SOURCE -g
CFLAGS_DEBUG = -I$(MPIpath)/include -c -g 
CFLAGS_TEST  = -I$(MPIpath)/include -c
CFLAGS_HDF5  = -I$(HDF5path)/include
CFLAGS_NCMPI = -I$(NCMPI_PATH)/include

#----------------------------------------------------------------------------
# Linker flags
#
#  There is a seperate version of the linker flags for each of the _OPT, 
#  _DEBUG, and _TEST cases.
#----------------------------------------------------------------------------

LFLAGS       =  -o  

#----------------------------------------------------------------------------
# Library specific linking
#
#  If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to
#  create a macro in this Makefile.h for LIB_xxx, which will be added to the
#  link line when FLASH is built.  This allows us to switch between different
#  (incompatible) libraries.  We also create a _OPT, _DEBUG, and _TEST
#  library macro to add any performance-minded libraries (like fast math),
#  depending on how FLASH was setup.
#----------------------------------------------------------------------------

LIB_MPI     =
LIB_HDF5    = -L$(HDF5path)/lib -lhdf5 -lz  

LIB_PGF90   =-L $/usr/pgi/linux86-64/2010/cuda/2.3/lib -lcudafor -L/usr/pgi/linux86-64/2010/cuda/2.3/lib -lcudart -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc
LIB_PAPI    = $(PAPI_PATH)/lib/libpapi.a $(PAPI_PATH)/lib/_fixunssfdi.o
LIB_PNG     = -lpng -lz

LIB_OPT     =
LIB_DEBUG   =
LIB_TEST    =

LIB_NCMPI   = -L$(NCMPI_PATH)/lib -lpnetcdf -L/usr/local/pgi-5.2.4/linux86/5.2/lib -lpgc

LIB_MPE     =
#----------------------------------------------------------------------------
# Additional machine-dependent object files
#
#  Add any machine specific files here -- they will be compiled and linked
#  when FLASH is built.
#----------------------------------------------------------------------------

MACHOBJ = 

#----------------------------------------------------------------------------
# Additional commands
#---------------------------------------------------------------------------- 

MV    = mv -f
AR    = ar -r
RM    = rm -f
CD    = cd
RL    = ranlib
ECHO  = echo

[MatColgrove]

Hi jmckennon,

What I really need to know is what link line your make produces. From the information given, I’m assuming that you’re linking with ‘mpif90’. This is just a wrapper script configured to use whatever compiler you used to build your MPI library with. If you have a PGI built MPI library available (we ship a debug version with the compilers located in the $PGI/linux86-64/2010/mpi directory), you might want to try using this instead.

Alternatively, I see that you have a “LIB_PGF90” variable defined in your makefile. First make sure that this is being added to your link and that the PATHs are correct. You also need to change “-L $/usr” to “-L/usr” and add a PATH to the PGI runtime libraries. For example “-L/usr/pgi/linux86-64/10.9/lib”. Be sure to change “/usr/pgi” to the actual location of your install and change “10.9” to the compiler version you’re using.

Hope this helps,
Mat

[jmckennon]

Hi Mat,

Thanks for the help with this! Here’s what’s produced with my make line:
Directly after the code below the undefined references begin. The pgi stuff is located at /opt/pgi/linux86/10.6/lib. This directory doesn’t have anything to do with cudafor though. The cudafor.mod is located at /opt/pgi/linux86/10.6/include.

I was attempting to add in the pgf90 runtime libraries into the makefile, usually though that line isn’t there.

[gravity@localhost object]$ gmake --jobs=8
rm -f reorder.sh
mpif90 -c -O2  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rieman.cuf
mpif90 -c -O2  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getArgument.F90
mpif90 -c -O2  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getOpt.F90
gfortran: rieman.cuf: linker input file unused because linking not done
Generating Build Statistics
Generating Buildstamp
./make_bstats
./make_bstamp
Generating Flash Release
./make_release
mpif90 -c -O2  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstamp.F90
mpif90 -c -O2  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_flashRelease.F90
mpicc -I/home/gravity/hdf5-1.6.10/hdf5/include -I/home/gravity/openmpi-1.4.2/ompi/include/include -c -O2 -D_LARGEFILE64_SOURCE -g -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstats.c
Linking in Units and Libraries
mpif90 -o   flash3 Burn.o Burn_computeDt.o Burn_finalize.o Burn_init.o Burn_interface.o Burn_nseAtDens.o Burn_nseAtPres.o Conductivity.o Conductivity_finalize.o Conductivity_init.o Conductivity_interface.o Cool.o Cool_computeDt.o Cool_finalize.o Cool_init.o Cool_interface.o Cool_unitTest.o Cosmology_cdmPowerSpectrum.o Cosmology_computeDt.o Cosmology_finalize.o Cosmology_getOldRedshift.o Cosmology_getParams.o Cosmology_getRedshift.o Cosmology_init.o Cosmology_interface.o Cosmology_massToLength.o Cosmology_redshiftHydro.o Cosmology_redshiftToTime.o Cosmology_sendOutputData.o Cosmology_solveFriedmannEqn.o Diffuse_computeDt.o Diffuse_finalize.o Diffuse_init.o Diffuse_interface.o Diffuse_species.o Diffuse_therm.o Diffuse_visc.o Driver_abortFlash.o Driver_abortFlashC.o Driver_checkMPIErrorCode.o Driver_computeDt.o Driver_data.o Driver_evolveFlash.o Driver_finalizeFlash.o Driver_finalizeSourceTerms.o Driver_getDt.o Driver_getElapsedWCTime.o Driver_getNStep.o Driver_getSimTime.o Driver_getTimeStamp.o Driver_init.o Driver_initFlash.o Driver_initMaterialProperties.o Driver_initParallel.o Driver_initSourceTerms.o Driver_interface.o Driver_putTimeStamp.o Driver_sendOutputData.o Driver_sourceTerms.o Driver_verifyInitDt.o Eos.o Eos_data.o Eos_finalize.o Eos_getData.o Eos_getParameters.o Eos_guardCells.o Eos_init.o Eos_interface.o Eos_putData.o Eos_wrapped.o Flash.o Gravity_accelAtCoords.o Gravity_accelListOfBlocks.o Gravity_accelOneRow.o Gravity_computeDt.o Gravity_finalize.o Gravity_init.o Gravity_interface.o Gravity_potentialListOfBlocks.o Grid_bcApplyToRegion.o Grid_bcApplyToRegionSpecialized.o Grid_computeUserVars.o Grid_conserveField.o Grid_conserveFluxes.o Grid_data.o Grid_dump.o Grid_fillGuardCells.o Grid_finalize.o Grid_getBlkBC.o Grid_getBlkBoundBox.o Grid_getBlkCenterCoords.o Grid_getBlkCornerID.o Grid_getBlkData.o Grid_getBlkIndexLimits.o Grid_getBlkPhysicalSize.o Grid_getBlkPtr.o Grid_getBlkRefineLevel.o Grid_getBlkType.o Grid_getCellCoords.o Grid_getDeltas.o Grid_getFluxData.o Grid_getGeometry.o Grid_getGlobalIndexLimits.o Grid_getListOfBlocks.o Grid_getLocalNumBlks.o Grid_getMaxCommonRefinement.o Grid_getMinCellSize.o Grid_getMyPE.o Grid_getNumProcs.o Grid_getPlaneData.o Grid_getPointData.o Grid_getProcGridInfo.o Grid_getRowData.o Grid_getSingleCellCoords.o Grid_getSingleCellVol.o Grid_init.o Grid_initDomain.o Grid_interface.o Grid_limitAbundance.o Grid_mapMeshToParticles.o Grid_mapParticlesToMesh.o Grid_markBlkDerefine.o Grid_markBlkRefine.o Grid_markRefineDerefine.o Grid_markRefineSpecialized.o Grid_moveParticles.o Grid_moveParticlesGlobal.o Grid_outsideBoundBox.o Grid_pfft.o Grid_pfftFinalize.o Grid_pfftGetIndexLimits.o Grid_pfftInit.o Grid_pfftMapFromOutput.o Grid_pfftMapToInput.o Grid_putBlkData.o Grid_putFluxData.o Grid_putLocalNumBlks.o Grid_putPlaneData.o Grid_putPointData.o Grid_putRowData.o Grid_releaseBlkPtr.o Grid_renormAbundance.o Grid_renormMassScalars.o Grid_restrictAllLevels.o Grid_restrictByLevels.o Grid_sendOutputData.o Grid_setFluxHandling.o Grid_solvePoisson.o Grid_sortParticles.o Grid_unitTest.o Grid_updateRefinement.o Heat.o Heat_computeDt.o Heat_finalize.o Heat_init.o Heat_interface.o Hydro.o Hydro_computeDt.o Hydro_data.o Hydro_detectShock.o Hydro_finalize.o Hydro_init.o Hydro_interface.o Hydro_sendOutputData.o IO_data.o IO_finalize.o IO_getPrevScalar.o IO_getScalar.o IO_init.o IO_initRPsFromCheckpoint.o IO_interface.o IO_output.o IO_outputFinal.o IO_outputInitial.o IO_readCheckpoint.o IO_readParticles.o IO_readUserArray.o IO_sendOutputData.o IO_setScalar.o IO_updateScalars.o IO_writeCheckpoint.o IO_writeIntegralQuantities.o IO_writeParticles.o IO_writePlotfile.o IO_writeUserArray.o Ionize.o Ionize_equil.o Ionize_finalize.o Ionize_init.o Ionize_interface.o Logfile_break.o Logfile_close.o Logfile_create.o Logfile_data.o Logfile_finalize.o Logfile_getDateTimeStr.o Logfile_init.o Logfile_interface.o Logfile_open.o Logfile_stamp.o Logfile_stampMessage.o Logfile_stampVarMask.o Logfile_writeSummary.o MagneticResistivity.o MagneticResistivity_finalize.o MagneticResistivity_init.o MagneticResistivity_interface.o MassDiffusivity.o MassDiffusivity_finalize.o MassDiffusivity_init.o MassDiffusivity_interface.o Multispecies_finalize.o Multispecies_getAvg.o Multispecies_getProperty.o Multispecies_getSum.o Multispecies_getSumFrac.o Multispecies_getSumInv.o Multispecies_getSumSqr.o Multispecies_init.o Multispecies_interface.o Multispecies_list.o Multispecies_setProperty.o Multispecies_unitTest.o Paramesh_finalize.o Paramesh_init.o Particles_accumCount.o Particles_advance.o Particles_computeDt.o Particles_dump.o Particles_finalize.o Particles_getCountPerBlk.o Particles_getGlobalNum.o Particles_getLocalNum.o Particles_init.o Particles_initData.o Particles_initForces.o Particles_initPositions.o Particles_interface.o Particles_longRangeForce.o Particles_mapFromMesh.o Particles_mapToMeshOneBlk.o Particles_putLocalNum.o Particles_sendOutputData.o Particles_shortRangeForce.o Particles_unitTest.o Particles_updateAttributes.o Particles_updateGridVar.o Particles_updateRefinement.o PhysicalConstants_data.o PhysicalConstants_get.o PhysicalConstants_init.o PhysicalConstants_interface.o PhysicalConstants_list.o PhysicalConstants_listUnits.o Profiler_getSummary.o Profiler_init.o Profiler_interface.o Profiler_start.o Profiler_stop.o RuntimeParameters_add.o RuntimeParameters_bcast.o RuntimeParameters_data.o RuntimeParameters_finalize.o RuntimeParameters_get.o RuntimeParameters_getAll.o RuntimeParameters_getNum.o RuntimeParameters_getNumIgn.o RuntimeParameters_getPrev.o RuntimeParameters_init.o RuntimeParameters_interface.o RuntimeParameters_mapStrToInt.o RuntimeParameters_read.o RuntimeParameters_set.o RuntimeParameters_setPrev.o RuntimeParameters_stampIgnored.o Simulation_customizeProlong.o Simulation_data.o Simulation_defineDomain.o Simulation_finalize.o Simulation_getRenormGroup.o Simulation_getVarnameType.o Simulation_init.o Simulation_initBlock.o Simulation_initRestart.o Simulation_initSpecies.o Simulation_interface.o Simulation_mapIntToStr.o Simulation_mapParticlesVar.o Simulation_mapStrToInt.o Simulation_sendOutputData.o Stir.o Stir_computeDt.o Stir_finalize.o Stir_init.o Stir_interface.o Timers_data.o Timers_finalize.o Timers_getSummary.o Timers_init.o Timers_interface.o Timers_reset.o Timers_start.o Timers_stop.o Utilities_findMean.o Viscosity.o Viscosity_finalize.o Viscosity_init.o Viscosity_interface.o amr_1blk_bcset.o amr_1blk_cc_cp_remote.o amr_1blk_cc_prol_gen_unk_fun.o amr_1blk_cc_prol_gen_work_fun.o amr_1blk_cc_prol_genorder.o amr_1blk_cc_prol_inject.o amr_1blk_cc_prol_linear.o amr_1blk_cc_prol_user.o amr_1blk_cc_prol_work_genorder.o amr_1blk_cc_prol_work_inject.o amr_1blk_cc_prol_work_linear.o amr_1blk_cc_prol_work_user.o amr_1blk_copy_soln.o amr_1blk_ec_cp_remote.o amr_1blk_ec_prol_gen_fun.o amr_1blk_ec_prol_genorder.o amr_1blk_ec_prol_linear.o amr_1blk_ec_prol_user.o amr_1blk_fc_clean_divb.o amr_1blk_fc_cp_remote.o amr_1blk_fc_prol_dbz.o amr_1blk_fc_prol_gen_fun.o amr_1blk_fc_prol_genorder.o amr_1blk_fc_prol_inject.o amr_1blk_fc_prol_linear.o amr_1blk_fc_prol_user.o amr_1blk_guardcell_reset.o amr_1blk_guardcell_srl.o amr_1blk_nc_cp_remote.o amr_1blk_nc_prol_gen_fun.o amr_1blk_nc_prol_genorder.o amr_1blk_nc_prol_linear.o amr_1blk_nc_prol_user.o amr_1blk_save_soln.o amr_1blk_t_to_perm.o amr_1blk_to_perm.o amr_bcset_init.o amr_block_geometry.o amr_close.o amr_get_new_nodetypes.o amr_initialize.o amr_mg_common.o amr_mg_init.o amr_mg_morton_process.o amr_mg_prolong.o amr_mg_restrict.o amr_mpi_find_blk_in_buffer.o amr_perm_to_1blk.o amr_prolong_cc_fun_init.o amr_prolong_face_fun_init.o amr_prolong_fun_init.o amr_prolong_gen_unk1_fun.o amr_prolong_gen_work1_fun.o amr_reorder_grid.o amr_restrict_ec_fun.o amr_restrict_ec_genorder.o amr_restrict_ec_user.o amr_restrict_edge.o amr_restrict_fc_fun.o amr_restrict_fc_genorder.o amr_restrict_fc_user.o amr_restrict_nc_fun.o amr_restrict_nc_genorder.o amr_restrict_nc_user.o amr_restrict_red.o amr_restrict_unk_fun.o amr_restrict_unk_genorder.o amr_restrict_unk_user.o amr_restrict_work_fun.o amr_restrict_work_fun_recip.o amr_restrict_work_genorder.o amr_restrict_work_user.o amr_set_runtime_parameters.o amr_system_calls.o avisco.o cma_flatten.o coeff.o concatStringWithInt.o constants.o current_date_time.o detect.o dr_set_rlimits.o dr_sleep.o eos_fillMapLookup.o eos_variableMap.o findWords.o flash_convert_cc_hook.o flash_unconvert_cc_hook.o flaten.o getType.o gr_amr_dump_runtime_parameters.o gr_bcApplyToAllBlks.o gr_bcApplyToOneFace.o gr_bcData.o gr_bcFinalize.o gr_bcGetRegion.o gr_bcInit.o gr_bcInterface.o gr_bcMapBcType.o gr_bcPutRegion.o gr_checkDataType.o gr_checkGridConsistency.o gr_checkGridState.o gr_commSetUp.o gr_computeMorton.o gr_conserveToPrimitive.o gr_createBlock.o gr_createDomain.o gr_enforceMaxRefine.o gr_ensureValidNeighborInfo.o gr_expandDomain.o gr_extendedGetCellCoords.o gr_extendedGetDeltas.o gr_findAllNeghID.o gr_findBlock.o gr_findMean.o gr_findNeghID.o gr_findWhichChild.o gr_findWhichChildren.o gr_flashHook_interfaces.o gr_freeCommRecvBuffer.o gr_getBlkHandle.o gr_getCellFaceArea.o gr_getCellVol.o gr_getDataOffsets.o gr_getIndex.o gr_getInteriorBlkPtr.o gr_hgFinalize.o gr_hgInit.o gr_hgInterface.o gr_hgMapBcType.o gr_hgPfftFinalize.o gr_hgPfftInit.o gr_hg_amr_1blk_bcset_work.o gr_initGeometry.o gr_initParameshArrays.o gr_interface.o gr_interfaceTypeDecl.o gr_isoInterface.o gr_isoMpoleFinalize.o gr_isoMpoleInit.o gr_makeMaskConsistent.o gr_markDerefineByTime.o gr_markEllipsoid.o gr_markInRadius.o gr_markInRectangle.o gr_markRefineDerefine.o gr_markVarThreshold.o gr_markWithRadius.o gr_mpoleFinalize.o gr_mpoleInit.o gr_packBCs.o gr_pfftFinalize.o gr_pfftInit.o gr_pfftInterface.o gr_pfftInterfaceTypeDecl.o gr_pfftPoissonDirect.o gr_primitiveToConserve.o gr_ptFillBlkParticleInfo.o gr_ptFinalize.o gr_ptInit.o gr_ptInterface.o gr_ptMapInit.o gr_ptMarkRefineDerefine.o gr_recreateDomain.o gr_releaseInteriorBlkPtr.o gr_restrictTree.o gr_sanitizeDataAfterInterp.o gr_searchNeighbor.o gr_setDataStructInfo.o gr_setGcFillNLayers.o gr_setMasks.o gr_solversFinalize.o gr_solversInit.o gr_unmarkRefineByLogRadius.o gr_updateData.o gr_updateRefinement.o grdvel.o hy_ppm_block.o hy_ppm_completeGeomFactors.o hy_ppm_force.o hy_ppm_geom.o hy_ppm_getTemporaryData.o hy_ppm_interface.o hy_ppm_putTemporaryData.o hy_ppm_sweep.o hy_ppm_updateSoln.o hydro_1d.o interp.o intrfc.o io.o io_bcastScalars.o io_closeFile.o io_comp_decomp.o io_finalizeListsRead.o io_finalizeListsWrite.o io_getAllScalars.o io_getNumScalars.o io_getOutputName.o io_getVarExtrema.o io_h5file_interface.o io_h5read_bflags.o io_h5read_blksize.o io_h5read_bndbox.o io_h5read_coords.o io_h5read_generic_int_arr.o io_h5read_generic_real_arr.o io_h5read_gid.o io_h5read_header.o io_h5read_lists.o io_h5read_lrefine.o io_h5read_nodetype.o io_h5read_unknowns.o io_h5read_which_child.o io_h5write_bflags.o io_h5write_blksize.o io_h5write_blksize_sp.o io_h5write_bndbox.o io_h5write_bndbox_sp.o io_h5write_coords.o io_h5write_coords_sp.o io_h5write_generic_int_arr.o io_h5write_generic_real_arr.o io_h5write_gid.o io_h5write_header.o io_h5write_lists.o io_h5write_lrefine.o io_h5write_nodetype.o io_h5write_procnumber.o io_h5write_unknowns.o io_h5write_unknowns_sp.o io_h5write_which_child.o io_initFile.o io_isPlotVar.o io_memoryReport.o io_memory_usage.o io_prepareListsRead.o io_prepareListsWrite.o io_prepareSimInfo.o io_ptCorrectNextPartTime.o io_ptInit.o io_ptInterface.o io_ptResetNextFile.o io_ptSendOutputData.o io_readData.o io_readRPsFromCheckpoint.o io_setPrevScalar.o io_writeData.o log_allocateStrArr.o log_getUnitsArr.o makeLowercase.o mesh_test.o monot.o mpi_amr_1blk_guardcell.o mpi_amr_1blk_guardcell_c_to_f.o mpi_amr_1blk_restrict.o mpi_amr_boundary_block_info.o mpi_amr_bsort.o mpi_amr_checkpoint.o mpi_amr_checkpoint_default.o mpi_amr_checkpoint_hdf5.o mpi_amr_checkpoint_mpiio.o mpi_amr_comm_setup.o mpi_amr_derefine_blocks.o mpi_amr_edge_average.o mpi_amr_edge_average_udt.o mpi_amr_edge_average_vdt.o mpi_amr_edge_diagonal_check.o mpi_amr_flux_conserve.o mpi_amr_flux_conserve_udt.o mpi_amr_flux_conserve_vdt.o mpi_amr_get_remote_block.o mpi_amr_get_remote_block_fvar.o mpi_amr_global_domain_limits.o mpi_amr_gsurr_blks.o mpi_amr_guardcell.o mpi_amr_local_surr_blks.o mpi_amr_morton.o mpi_amr_morton_limits.o mpi_amr_plotfile_chombo.o mpi_amr_prolong.o mpi_amr_prolong_fc_divbconsist.o mpi_amr_redist_blk.o mpi_amr_refine_blocks.o mpi_amr_refine_derefine.o mpi_amr_restrict.o mpi_amr_restrict_bnd_data.o mpi_amr_restrict_bnd_data_vdt.o mpi_amr_restrict_edge_data.o mpi_amr_restrict_edge_data_vdt.o mpi_amr_singular_line.o mpi_amr_store_comm_info.o mpi_amr_store_comm_info_mg.o mpi_amr_test_neigh_values.o mpi_amr_tree_setup.o mpi_get_buffer.o mpi_get_edge_buffer.o mpi_get_flux_buffer.o mpi_lib.o mpi_mort_comm_for_surrblks.o mpi_morton.o mpi_morton_bnd.o mpi_morton_bnd_fluxcon.o mpi_morton_bnd_prolong1.o mpi_morton_bnd_restrict.o mpi_pack_blocks.o mpi_pack_edges.o mpi_pack_fluxes.o mpi_pack_tree_info.o mpi_put_buffer.o mpi_put_edge_buffer.o mpi_put_edge_buffer_1blk.o mpi_put_flux_buffer.o mpi_set_message_limits.o mpi_set_message_limits_unpack.o mpi_set_message_sizes.o mpi_unpack_blocks.o mpi_unpack_edges.o mpi_unpack_fluxes.o mpi_unpack_tree_info.o mpi_wrapper_dble.o mpi_wrapper_int.o mpi_wrapper_logical.o mpi_wrapper_real.o nameSyntaxError.o nameValueLL_bcast.o nameValueLL_data.o nameValueLL_get.o nameValueLL_getAll.o nameValueLL_getNum.o nameValueLL_logRules.o nameValueLL_rules.o nameValueLL_set.o paramesh_comm_data.o paramesh_dimensions.o paramesh_interfaces.o paramesh_mpi_interfaces.o pc_utilities.o physicaldata.o prolong_arrays.o quicksort_index.o rationalize_list.o removeNullChar.o rieman.o riemann_hlle.o rp_getArgument.o rp_getOpt.o rp_initParameters.o rp_rules.o rp_storeIgnoredParams.o setup_buildstamp.o setup_buildstats.o setup_flashRelease.o setup_getFlashUnits.o states.o timings.o tmr_buildSummary.o tmr_create.o tmr_etime.o tmr_findTimerIndex.o tmr_getMaxCallStackDepth.o tmr_getMaxTimerParents.o tmr_init.o tmr_lookupIndex.o tmr_stackLib.o tree.o umap.o user_coord_transfm.o ut_conversionInterface.o ut_convertToArrayIndicies.o ut_convertToMemoryOffset.o ut_fndpos.o ut_hunt.o ut_interpolationInterface.o ut_polint.o ut_quadraticInterpol.o workspace.o -L/home/gravity/hdf5-1.6.10/hdf5/lib -lhdf5 -lz

[MatColgrove]

mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rieman.cuf
mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getArgument.F90
mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getOpt.F90
gfortran: rieman.cuf: linker input file unused because linking not done
Generating Build Statistics
Generating Buildstamp

Looks like your mpif90 is configured to use gfortran. My recommendation would be to build OpenMPI with PGI (Account Login | PGI), point your PATH to this new build, and then try building FLASH again. Be sure to add the “-Mcuda” flag to your link options so the CUDA libraries will be added.

• Mat

[jmckennon]

Hi Mat,

I apologize if this is a trivial question, but I’m not sure what you mean by “point your path to this new build.” How would I do this? I appreciate the help alot!

[MatColgrove]

“point your path to this new build.”

Sorry, I should have said that you should include the fully qualified path name to the new PGI built MPI installation’s bin directory to your shells’s PATH environment variable. For example in csh:

setenv PATH /path/to/my/openmpi/bin:$PATH

or in bash

export PATH=/path/to/my/openmpi/bin:$PATH

Note you can also use the pre-compiled MPICH version that accompanies the compilers. It’s an optional install so may or may not have been installed with the compiler. Also, it is a debug version meant for use the PGI debugger PGDBG, but may at least get you going until you can build an optimized OpenMPI.

% setenv PATH /usr/pgi/linux86-64/2010/mpi/mpich/bin/:$PATH
% which mpif90
/usr/pgi/linux86-64/2010/mpi/mpich/bin//mpif90
% mpif90 -V
mpif90 for 1.0.5

pgf90 10.9-0 64-bit target on x86-64 Linux -tp nehalem-64
Copyright 1989-2000, The Portland Group, Inc.  All Rights Reserved.
Copyright 2000-2010, STMicroelectronics, Inc.  All Rights Reserved.

Hope this helps,
Mat

[jmckennon]

Hi Mat,

I followed the instructions to install openmpi with pgi but it didn’t appear to work. I type in the command env CC=pgcc FC=pgfortran F77=pgfortran CXX=pgcpp CFLAGS=-fast FCFLAGS=-fast FFLAGS=-fast CXXFLAGS=-fast ./configure --prefix=/usr/local/openmpi > & configure.log

and it writes in the configure.log “Error C Compiler cannot create executables.” I tried changins the CC command to use gcc just to get it to work and it still said the same thing. Do you know why this is? Thanks for the help, you’re a lifesaver.

[MatColgrove]

One possibility is the OpenMPI version you’re using. Earlier version of OpenMPI have a bug which recognize PGI version “10.x” as “1.0x” and use the wrong flags. (See problem compiling openmpi-1.3.2 with PG Compiles) Make sure that your building OpenMPI 1.4.1 or later.

Also, try setting each of the environment variables before running configure. It’s possible that the env settings aren’t being used.

i.e. in Bash:

export CC=pgcc 
export FC=pgfortran 
export F77=pgfortran 
export CXX=pgcpp 
export CFLAGS=-fast 
export FCFLAGS=-fast 
export FFLAGS=-fast 
export CXXFLAGS=-fast 
./configure --prefix=/usr/local/openmpi > & configure.log

in csh replace ‘export’ with ‘setenv’ and remove the “=”.


Finally, please double check that pgcc is working properly by compiling a simple C program.

If none of these fix the problem, please post the full error from the config.log file (i.e the log file created by the configure script, not the redirected output from configure.log)

• Mat

[jmckennon]

Hi Mat,

In the config.log file I think I found where the error is, although I’m not sure why it is occurring or how to fix it. The code is posted below. I can post the entire config.log if this does not provide enough information. The code I used to compile it was identical to the bash script you provided. Thanks for the help!!!

configure:5672: result: no
configure:5685: checking for package/brand string
configure:5701: result: Open MPI gravity@localhost.localdomain Distribution
configure:5707: checking for ident string
configure:5724: result: 1.4.2
configure:5731: checking whether to add padding to the openib control header
configure:5744: result: no
configure:5758: checking whether to use an alternative checksum algo for messages
configure:5771: result: no
configure:5993: checking for style of include used by make
configure:6021: result: GNU
configure:6096: checking for gcc
configure:6123: result: pgcc
configure:6355: checking for C compiler version
configure:6363: pgcc --version >&5
pgcc-Error-Unknown switch: --version
configure:6367: $? = 1
configure:6374: pgcc -v >&5
pgcc-Warning-No files to process
configure:6378: $? = 0
configure:6385: pgcc -V >&5

pgcc 10.5-0 64-bit target on x86-64 Linux -tp gh-64
Copyright 1989-2000, The Portland Group, Inc.  All Rights Reserved.
Copyright 2000-2010, STMicroelectronics, Inc.  All Rights Reserved.
configure:6389: $? = 0
configure:6412: checking for C compiler default output file name
configure:6434: pgcc -DNDEBUG -fast   conftest.c  >&5
pgi-cc-lin64: LICENSE MANAGER PROBLEM: No such feature exists.

Feature:       pgi-cc-lin64
License path:  /opt/pgi/license.dat:
FLEXnet Licensing error:-5,357
For further information, refer to the FLEXnet Licensing documentation,
available at "www.acresso.com".
configure:6438: $? = 2
configure:6476: result:
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "Open MPI"
| #define PACKAGE_TARNAME "openmpi"
| #define PACKAGE_VERSION "1.4.2"
| #define PACKAGE_STRING "Open MPI 1.4.2"
| #define PACKAGE_BUGREPORT "http://www.open-mpi.org/community/help/"
| #define OMPI_MAJOR_VERSION 1
| #define OMPI_MINOR_VERSION 4
| #define OMPI_RELEASE_VERSION 2
| #define OMPI_GREEK_VERSION ""
| #define OMPI_VERSION "2"
| #define OMPI_RELEASE_DATE "May 04, 2010"
| #define ORTE_MAJOR_VERSION 1
| #define ORTE_MINOR_VERSION 4
| #define ORTE_RELEASE_VERSION 2
| #define ORTE_GREEK_VERSION ""
| #define ORTE_VERSION "2"
| #define ORTE_RELEASE_DATE "May 04, 2010"
| #define OPAL_MAJOR_VERSION 1
| #define OPAL_MINOR_VERSION 4
| #define OPAL_RELEASE_VERSION 2
| #define OPAL_GREEK_VERSION ""
| #define OPAL_VERSION "2"
| #define OPAL_RELEASE_DATE "May 04, 2010"
| #define OMPI_ARCH "x86_64-unknown-linux-gnu"
| #define OMPI_ENABLE_MEM_DEBUG 0
| #define OMPI_ENABLE_MEM_PROFILE 0
| #define OMPI_ENABLE_DEBUG 0
| #define OMPI_GROUP_SPARSE 0
| #define OMPI_WANT_MPI_CXX_SEEK 1
| #define MPI_PARAM_CHECK ompi_mpi_param_check
| #define OMPI_WANT_PRETTY_PRINT_STACKTRACE 1
| #define OMPI_WANT_PERUSE 0
| #define OMPI_ENABLE_PTY_SUPPORT 1
| #define OMPI_ENABLE_HETEROGENEOUS_SUPPORT 0
| #define OPAL_ENABLE_TRACE 0

| #define ORTE_DISABLE_FULL_SUPPORT 0
| #define OPAL_ENABLE_FT 0
| #define OPAL_ENABLE_FT_CR 0
| #define OMPI_WANT_HOME_CONFIG_FILES 1
| #define OPAL_ENABLE_IPV6 1
| #define ORTE_WANT_ORTERUN_PREFIX_BY_DEFAULT 0
| #define OPAL_PACKAGE_STRING "Open MPI gravity@localhost.localdomain Distribution"
| #define OPAL_IDENT_STRING "1.4.2"
| #define OMPI_OPENIB_PAD_HDR 0
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:6482: error: in `/home/gravity/openmpi-1.4.2':
configure:6485: error: C compiler cannot create executables
See `config.log' for more details.

[jmckennon]

Hi Mat,

I also get this error when I try to run a simple C program


[gravity@localhost openmpi-1.4.2]$ pgcc hello.c
pgi-cc-lin64: LICENSE MANAGER PROBLEM: No such feature exists.
Feature: pgi-cc-lin64
License path: /opt/pgi/license.dat:
FLEXnet Licensing error:-5,357
For further information, refer to the FLEXnet Licensing documentation,
available at “www.acresso.com”.

[jmckennon]

Hi Mat,

I was able to configure and install the openmpi with pgfortran but now my mpif90 doesn’t work. I had to use gcc and g++ instead of pgcc and pgcpp because we don’t have the licenses for those. When I run my gmake command now I get the error

---

Unfortunately, this installation of Open MPI was not compiled with
Fortran 90 support. As such, the mpif90 compiler is non-functional.

gmake: *** [rieman.o] Error 1
gmake: *** Waiting for unfinished jobs…

Unfortunately, this installation of Open MPI was not compiled with
Fortran 90 support. As such, the mpif90 compiler is non-functional.

gmake: *** [rp_getOpt.o] Error 1

Unfortunately, this installation of Open MPI was not compiled with
Fortran 90 support. As such, the mpif90 compiler is non-functional.

gmake: *** [rp_getArgument.o] Error 1


I have no idea what this means. Any ideas?

[MatColgrove]

I had to use gcc and g++ instead of pgcc and pgcpp because we don’t have the licenses for those.

This should be fine. We’re object compatible with GNU.

I found a post in the OpenMPI forums that may be useful: http://www.open-mpi.org/community/lists/users/2009/05/9395.php. Can you please post your environment variables, as well as the configure, make and make install command you used?

• Mat

[jmckennon]

Hi Mat,

I’m finally making some progress. Openmpi is now installed and mpif90 works. I set FC and F77 to use pgfortran. Now, the program compiles up until the file calls the “use cudafor” line. It gives this error :

mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rieman.F90
mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getArgument.F90
mpif90 -c -O2 -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getOpt.F90
Generating Buildstamp
./make_bstamp
rieman.F90:91.11:

use cudafor
1
Fatal Error: File ‘cudafor.mod’ opened at (1) is not a GFORTRAN module file
gmake: *** [rieman.o] Error 1
gmake: *** Waiting for unfinished jobs…
Generating Build Statistics
./make_bstats

Why is it telling me it’s not a GFORTRAN module? It’s not being compiled with gfortran at all?

I copied the cudafor.mod file from the include directory to the local directory I run the make command from because I was getting a “file not found” problem. Also, the name of the file that I’m re-writing is rieman.F90. When I change the extension to .cuf it doesn’t work. Will it still accept the cuda-language stuff if the extension is .F90?

[MatColgrove]

Why is it telling me it’s not a GFORTRAN module? It’s not being compiled with gfortran at all?

The only way to get this error message would be if you accidentally built with gfortran. Sorry, you’ll need to try again.

We do ship a MPICH build with the PGI compilers (though it is an optional install). What I would suggest is to do skip using OpenMPI for now and try changing ‘mpif90’ to ‘pgf90 -Mmpi -Mcuda’. You’ll be able to make progress and then later you can double back to using OpenMPI.

• Mat

[jmckennon]

Hi Mat,

I set the program to use the MPICH version that comes with pgfortran. When I run the program now I get:


rm -f reorder.sh
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rieman.F90
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getArgument.F90
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getOpt.F90
Generating Buildstamp
./make_bstamp
Generating Build Statistics
./make_bstats
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstamp.F90
Generating Flash Release
./make_release
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_flashRelease.F90
mpicc -I/home/gravity/hdf5-1.6.10/hdf5/include -I/opt/pgi/linux86-64/2010/mpi/mpich/include -c -O2 -D_LARGEFILE64_SOURCE -g -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstats.c
pgi-cc-lin64: LICENSE MANAGER PROBLEM: No such feature exists.
Feature: pgi-cc-lin64
License path: /opt/pgi/license.dat:
FLEXnet Licensing error:-5,357
For further information, refer to the FLEXnet Licensing documentation,
available at “www.acresso.com”.


It looks like it’s trying to call pgicc (which we don’t have a license for) instead of mpicc. Do I need to get a license for pgi C and CPP compilers to use this version of MPICH?

[jmckennon]

AH, nevermind I had my PATH pointing to the mpich directory, not the regular directory. Changing this fixed that error. I think I only have one problem left. I appreciate all the help.

gmake --jobs=8
rm -f reorder.sh
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rieman.F90
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getArgument.F90
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 rp_getOpt.F90
Generating Buildstamp
./make_bstamp
Generating Build Statistics
Generating Flash Release
./make_bstats
./make_release
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstamp.F90
mpicc -I/home/gravity/hdf5-1.6.10/hdf5/include -I/usr/local/include -c -O2 -D_LARGEFILE64_SOURCE -g -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_buildstats.c
pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_flashRelease.F90
PGF90-F-0004-Corrupt or Old Module file ./hydro_data.mod (rieman.F90: 114)
PGF90/x86-64 Linux 10.6-0: compilation aborted
gmake: *** [rieman.o] Error 2
gmake: *** Waiting for unfinished jobs…
/opt/pgi/linux86-64/10.6/lib/f90main.o: In function main': f90main.c:(.text+0x3c): undefined reference to MAIN_’
/opt/pgi/linux86-64/10.6/lib/f90main.o: In function main': f90main.c:(.text+0x3c): undefined reference to MAIN_’
gmake: *** [rp_getArgument.o] Error 2
/opt/pgi/linux86-64/10.6/lib/f90main.o: In function main': f90main.c:(.text+0x3c): undefined reference to MAIN_’
gmake: *** [setup_buildstamp.o] Error 2
/tmp/pgf90DEFdxqfKrpl1.o: In function rp_getopt_': /home/gravity/FLASH3.2/object/./rp_getOpt.F90:78: undefined reference to rp_getargument_’
/home/gravity/FLASH3.2/object/./rp_getOpt.F90:78: undefined reference to rp_getargument_' /opt/pgi/linux86-64/10.6/lib/f90main.o: In function main’:
f90main.c:(.text+0x3c): undefined reference to `MAIN_’
gmake: *** [rp_getOpt.o] Error 2
gmake: *** [setup_flashRelease.o] Error 2
)


I’m not sure where the “corrupt or old module” stuff is coming from. Doing a ls -a in the directory shows only one hydro_data.mod file in there. Is there a way to fix this?

Best,
Justin

[MatColgrove]

pgf90 -Mmpi -Mcuda -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 setup_flashRelease.F90
PGF90-F-0004-Corrupt or Old Module file ./hydro_data.mod (rieman.F90: 114)
PGF90/x86-64 Linux 10.6-0: compilation aborted

Most likely the hydro_data source file didn’t get rebuilt with PGI so the module file is in gfortran’s format. Try doing a make clean first so that all objects get rebuilt.

• Mat

[jmckennon]

Hi Mat,

That seemed to do the trick! All the libraries and everything appear to be set up appropriately. The only error I get now when I make the program is

[gravity@localhost object]$ gmake --jobs=8
rm -f reorder.sh
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 ut_interpolationInterface.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 Burn_interface.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 io.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 Grid_data.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 constants.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 Eos_data.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 Heat_interface.F90
pgf90 -Mmpi -Mcuda  -DMAXBLOCKS=200 -DNXB=8 -DNYB=8 -DNZB=8 -DN_DIM=3 Timers_data.F90
PGF90-W-0435-Array declared with zero size (Eos_data.F90: 52)
  0 inform,   1 warnings,   0 severes, 0 fatal for eos_data
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [ut_interpolationInterface.o] Error 2
gmake: *** Waiting for unfinished jobs....
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [Burn_interface.o] Error 2
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main'gmake: *** [constants.mod] Error 2
:
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [Heat_interface.o] Error 2
gmake: *** [Grid_data.o] Error 2
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [Eos_data.o] Error 2
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [io.o] Error 2
/opt/pgi/linux86-64/10.5/lib/f90main.o: In function `main':
f90main.c:(.text+0x3c): undefined reference to `MAIN_'
gmake: *** [Timers_data.o] Error 2

I’m expecting there to be errors related to the cudafortran coding I have done because I haven’t finished writing the code yet (wanted to get everything all set up before I did). Would this be related to that or is it something else?