I am trying to compile scipy 0.7.0 on the cray linux 2.2 nodes on the NICS kraken system. I’ve successfully built lapack, ATLAS, python 2.6.4, and numpy 1.3.0 but am having problems with scipy. Running “python setup.py install” in the scipy source directory yields the following error:
…
compiling C sources
C compiler: cc -DNDEBUG -fastsse -fPIC
creating build/temp.linux-x86_64-2.6/build
creating build/temp.linux-x86_64-2.6/build/src.linux-x86_64-2.6
creating build/temp.linux-x86_64-2.6/build/src.linux-x86_64-2.6/scipy
creating build/temp.linux-x86_64-2.6/build/src.linux-x86_64-2.6/scipy/fftpack
compile options: ‘-Ibuild/src.linux-x86_64-2.6 -I/lustre/scratch/jvanclev/opt/lib/python2.6/site-packages/numpy/core/include -I/lustre/scratch/jvanclev/opt/include/python2.6 -c’
cc: build/src.linux-x86_64-2.6/fortranobject.c
/opt/cray/xt-asyncpe/3.3/bin/cc: INFO: linux target is being used
cc: scipy/fftpack/src/zfftnd.c
/opt/cray/xt-asyncpe/3.3/bin/cc: INFO: linux target is being used
PGC-W-0156-Type not specified, ‘int’ assumed (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0039-Use of undeclared variable caches_zfftnd_fftpack (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0155-Pointer value created from a nonlong integral type (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0155-Pointer value created from a nonlong integral type (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0155-Pointer value created from a nonlong integral type (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0095-Type cast required for this conversion (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-W-0155-Pointer value created from a nonlong integral type (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0054-Subscript operator () applied to non-array (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-S-0059-Struct or union required on left of . or → (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC-F-0008-Error limit exceeded (scipy/fftpack/src/zfftnd_fftpack.c: 21)
PGC/x86-64 Linux 9.0-3: compilation aborted
…
I was unable to find any information on a problem in scipy with this source, so I was wondering whether there might be something specific to compiling with pgcc. Any thoughts?
That’s something I looked for, but I couldn’t find a problem with the code. Here is the relevant code from
zfftnd_fftpack.c (line 21 is the last line quoted below):
…
GEN_CACHE(zfftnd_fftpack, (int n, int rank)
, complex_double * ptr; int *iptr; int rank;
, ((caches_zfftnd_fftpack_.n == n)
&& (caches_zfftnd_fftpack.rank == rank))
, caches_zfftnd_fftpack[id].n = n;
caches_zfftnd_fftpack[id].ptr =
(complex_double *) malloc(2 * sizeof(double) * n);
caches_zfftnd_fftpack[id].iptr =
(int *) malloc(4 * rank * sizeof(int));
,
free(caches_zfftnd_fftpack[id].ptr);
free(caches_zfftnd_fftpack[id].iptr);
, 10)
…
And here is the relevant code from the header (fftpack.h):
… define GEN_CACHE(name,CACHEARG,CACHETYPE,CHECK,MALLOC,FREE,CACHESIZE)
typedef struct {
int n;
CACHETYPE
} cache_type_##name;
static cache_type_##name caches_##name[CACHESIZE];
static int nof_in_cache_##name = 0;
static int last_cache_id_##name = 0;
static int get_cache_id_##name CACHEARG {
int i,id = -1;
for (i=0;i<nof_in_cache_##name;i++)
if (CHECK) {
id=i;
break;
}
if (id>=0) goto exit;
if (nof_in_cache_##name<CACHESIZE) {
id = nof_in_cache_##name++;
} else {
id = (last_cache_id_##name<CACHESIZE-1)?last_cache_id_##name+1:0;
/fprintf(stderr,“Removing cache item n=%d\n”,caches_##name[id].n);/
FREE
caches_##name[id].n = 0;
}
/fprintf(stderr,“New cache item n=%d\n”,n);/
caches_##name[id].n = n;
MALLOC
exit:
last_cache_id_##name = id;
return id;
}
static void destroy_##name##caches(void) {
int id;
for (id=0;id<nof_in_cache##name;++id) {
FREE
caches_##name[id].n = 0;
}
nof_in_cache_##name = last_cache_id_##name = 0;
}
…_
I’ve sent this error on to our engineers for further investigation (TPR#16377). It appears to me to be a preprocessing bug where the line "static cache_type_##name caches_##name[CACHESIZE];" is getting processed to "static cache_type_zffnd_fftpackcaches_zffnd_fftpack[CACHESIZE];" (no space between fftpack and caches).
For reasons I don’t fully understand, the error only occurs when "static void destroy_##name##caches(void) {" is also used in the GEN_CACHE macro. If you change this line to "static void destroy_caches##name(void) {", then zfftnd.c will compile.
Thanks for looking into this! That does sound weird. Any suggestions for a short term fix or do you think I should just wait until the engineers have a chance to take a look?
While not ideal, I think the easiest thing to do would be to first preprocess the file (“pgcc -P zfftnd.c”). Next, edit the resulting “.i” file and add the missing space. Finally, compile the edited “.i” file (“pgcc -fast -c zfftnd.i”).
I’ve now been able to compile zfftnd.c, but the scipy compilation fails at another step:
…
building ‘scipy.spatial.ckdtree’ extension
compiling C sources
C compiler: cc -DNDEBUG -fastsse -fPIC
compile options: ‘-I/lustre/scratch/jvanclev/opt/lib/python2.6/site-packages/numpy/core/include -I/lustre/scratch/jvanclev/opt/include/python2.6 -c’
cc: scipy/spatial/ckdtree.c
/opt/cray/xt-asyncpe/3.3/bin/cc: INFO: linux target is being used
PGC-W-0106-Shift count out of range (scipy/spatial/ckdtree.c: 220)
PGC/x86-64 Linux 9.0-3: compilation completed with warnings
cc -shared build/temp.linux-x86_64-2.6/scipy/spatial/ckdtree.o -Lbuild/temp.linux-x86_64-2.6 -o build/lib.linux-x86_64-2.6/scipy/spatial/ckdtree.so
/opt/cray/xt-asyncpe/3.3/bin/cc: INFO: linux target is being used
/usr/bin/ld: build/temp.linux-x86_64-2.6/scipy/spatial/ckdtree.o: relocation R_X86_64_PC32 against __Pyx_ConsumeWhitespace' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value /opt/cray/xt-asyncpe/3.3/bin/cc: INFO: linux target is being used /usr/bin/ld: build/temp.linux-x86_64-2.6/scipy/spatial/ckdtree.o: relocation R_X86_64_PC32 against __Pyx_ConsumeWhitespace’ can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
error: Command “cc -shared build/temp.linux-x86_64-2.6/scipy/spatial/ckdtree.o -Lbuild/temp.linux-x86_64-2.6 -o build/lib.linux-x86_64-2.6/scipy/spatial/ckdtree.so” failed with exit status 2
…
I’m trying to recreate the problem here but need a bit of help. I’m able to get the build to use pgcc instead of gcc by setting CC in my environment. However, I’m not able to override the C flags it’s using. How did you work around this issue?
So far as I understand it, running “python setup.py install” defaults to the compiler flags that were used to compile python. When I compiled python, I used the following configure command:
./configure --prefix=/lustre/scratch/jvanclev/opt CC=cc CXX=CC OPT=-fastsse LINKFORSHARED=-Wl,–export-dynamic
These options get saved in a master Makefile in /usr/lib/python2.6/config/Makefile (or where ever your python installation is); presumably, you can edit this file to change the compile options, but I haven’t tried this.
In case it helps, here is the section of this Makefile from my python installation on the cray system that deals with the C compiler
…
=== Variables set by configure
VERSION= 2.6
srcdir= .
CC= cc
CXX= CC
MAINCC= $(CC)
LINKCC= $(PURIFY) $(MAINCC)
AR= ar
RANLIB= ranlib
SVNVERSION= echo Unversioned directory
Shell used by make (some versions default to the login shell, which is bad)
SHELL= /bin/sh
Use this to make a link between python$(VERSION) and python in $(BINDIR)
Now that I’m back in the office, I was able to recreate the error. Turns out to be another compiler problem where pgcc is removing what it thinks is a vestigial function. I have sent a report to our engineers (TPR#16393) and hopefully we can have this fixed soon.
In the mean time, you can add the internal compiler flag “-Mx,2,0x1000” to the compilation of ckdtree.c as well as vonmises_cython.c. " -Mx,2,0x1000" tells the compiler to not perform vestigial elimination. Note that internal flags can and do change from release to release.
Your advice worked perfectly; I was able to successfully compile scipy. Within python, I ran some the numpy and scipy tests, many of which failed. I’m going to look into it and see if the problem isn’t my compilation of lapack/ATLAS. I’ll post again if something comes up related to PGI.