Hi,
I’m using tensorflow_20.01-tf2-py3 to build an image for development of Tensorflow-based classification and regression algorithms. I’m having an interesting issue. When I start the baseline image (not even extending it yet) with docker, everything works fine and I can run Numpy and Tensorflow as expected.
But when I use singularity:
$ singularity exec tensorflow_20.01-tf2-py3.sif /bin/bash
$ python3
Python 3.6.9 (default, Nov 7 2019, 10:44:02)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/numpy/__init__.py", line 142, in <module>
from . import core
File "/usr/local/lib/python3.6/dist-packages/numpy/core/__init__.py", line 67, in <module>
raise ImportError(msg.format(path))
ImportError: Something is wrong with the numpy installation. While importing we detected an older version of numpy in ['/usr/local/lib/python3.6/dist-packages/numpy']. One method of fixing this is to repeatedly uninstall numpy until none is found, then reinstall this version.
Somehow the image, when run in singularity, has issues with loading numpy (and therefore TF as well). I’m at a bit of a loss on what to do: from all my exploring of the filesystem, I can only see a single numpy installation at /usr/local/lib/python3.6/dist-packages/numpy.
I’ve tried running this on multiple different machines, ranging from an Ubuntu 18.04 machine, to a CentOS7 machine, to my local OSX 10.15 laptop, and I get the same error regardless of the host.
This also seems to be broken in 19.12, but was fine in 19.11.
Can anyone provide any tips on how to get around this? Thanks for your help!