All,
I’m attempting to create a conda (miniconda) environment with tensorflow and rasterio. I can get rasterio to install in a conda environment but not the base python (/usr/bin/python3) and I am unable to get tensorflow to install inside a conda environment. I have tried lots of things, and the sudo pip3 install instructions found here: How to Install JetPack :: NVIDIA JetPack Documentation inform me that it is already satisfied, but when I launch python 3.9 from the conda enviro, it is not installed and will not import.
I’m at my wits end. I’ve gotten the system up and running with very few problems (a few) but can’t sort out how to get this going. I don’t want to flash and start all over if I don’t have to, but now tensorflow will not import in the 3.6 /usr/bin/python3 python either… Illegal Instruction core dump error, so presumeably I broke something trying to get it to install tf into the conda environment.
And then there was pip3…
It turns out that I was overlooking the fact that when I call pip3 as sudo pip3, it is not calling the anaconda installed pip3. This is how I sorted it out:
(geo) <user>@agx:~$ which -a pip
/home/<user>/miniforge3/envs/geo/bin/pip
/usr/local/bin/pip
(geo) <user>@agx:~$ which -a pip3
/home/<user>/miniforge3/envs/geo/bin/pip3
/usr/local/bin/pip3
/usr/bin/pip3
(geo) <user>@agx:~$ which pip3
/home/<user>/miniforge3/envs/geo/bin/pip3
(geo) <user>@agx:~$ sudo which pip3
[sudo] password for <user>:
/usr/local/bin/pip3
(geo) <user>@agx:~$ sudo /home/<user>/miniforge3/envs/geo/bin/pip3 install --pre --no-cache-dir --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow
Looking in indexes: https://pypi.org/simple, https://developer.download.nvidia.com/compute/redist/jp/v46
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
(geo) <user>@agx:~$ conda install python=3.6
...
(geo) <user>@agx:~$ sudo /home/<user>/miniforge3/envs/geo/bin/pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow
I included the full path to the desired pip3 (anaconda) and it seems to be installed now.
But now there is a new problem:
(geo) seth@agx:~$ python
Python 3.6.13 | packaged by conda-forge | (default, Feb 19 2021, 05:46:38)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2021-08-26 13:43:20.500471: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.10.2
/home/seth/miniforge3/envs/geo/lib/python3.6/site-packages/h5py/__init__.py:40: UserWarning: h5py is running against HDF5 1.10.6 when it was built against 1.10.0, this may cause problems
'{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.0, library is 1.10.6
SUMMARY OF THE HDF5 CONFIGURATION
=================================
General Information:
-------------------
HDF5 Version: 1.10.6
Configured on: Sat Jan 9 02:03:46 UTC 2021
Configured by: conda@2af0a685c269
Host system: aarch64-conda-linux-gnu
Uname information: Linux 2af0a685c269 4.15.0-36-generic #39~16.04.1-Ubuntu SMP Tue Sep 25 09:00:05 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
Byte sex: little-endian
Installation point: /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh
Compiling Options:
------------------
Build Mode: production
Debugging Symbols: no
Asserts: no
Profiling: no
Optimization Level: high
Linking Options:
----------------
Libraries: static, shared
Statically Linked Executables:
LDFLAGS: -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib -Wl,-rpath-link,/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib -L/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib
H5_LDFLAGS:
AM_LDFLAGS: -L/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib
Extra libraries: -lcrypto -lcurl -lpthread -lz -ldl -lm
Archiver: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-ar
AR_FLAGS: cr
Ranlib: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-ranlib
Languages:
----------
C: yes
C Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-cc
CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include
H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API
AM_CPPFLAGS: -I/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include
C Flags: -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/work=/usr/local/src/conda/hdf5_split-1.10.6 -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix
H5 C Flags: -std=c99 -pedantic -Wall -Wextra -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual -Wconversion -Wdeclaration-after-statement -Wdisabled-optimization -Wfloat-equal -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-enum -Wswitch-default -Wundef -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings -Wlogical-op -Wlarger-than=2560 -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wnull-dereference -Wunused-const-variable -Wduplicated-cond -Whsa -Wnormalized -Walloc-zero -Walloca -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -fstdarg-opt -s -Wno-inline -Wno-aggregate-return -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-suggest-attribute=const -Wno-suggest-attribute=pure -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
AM C Flags:
Shared C Library: yes
Static C Library: yes
Fortran: yes
Fortran Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-gfortran
Fortran Flags:
H5 Fortran Flags: -pedantic -Wall -Wextra -Wunderflow -Wimplicit-interface -Wsurprising -Wno-c-binding-type -s -O2
AM Fortran Flags:
Shared Fortran Library: yes
Static Fortran Library: yes
C++: yes
C++ Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-c++
C++ Flags: -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/work=/usr/local/src/conda/hdf5_split-1.10.6 -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix
H5 C++ Flags: -pedantic -Wall -W -Wundef -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor -Wctor-dtor-privacy -Wabi -finline-functions -s -O
AM C++ Flags:
Shared C++ Library: yes
Static C++ Library: yes
Java: no
Features:
---------
Parallel HDF5: no
Parallel Filtered Dataset Writes: no
Large Parallel I/O: no
High-level library: yes
Build HDF5 Tests: yes
Build HDF5 Tools: yes
Threadsafety: yes
Default API mapping: v110
With deprecated public symbols: yes
I/O filters (external): deflate(zlib)
MPE: no
Direct VFD: no
(Read-Only) S3 VFD: yes
(Read-Only) HDFS VFD: no
dmalloc: no
Packages w/ extra debug output: none
API tracing: no
Using memory checker: yes
Memory allocation sanity checks: no
Function stack tracing: no
Strict file format checks: no
Optimization instrumentation: no
Bye...
Aborted (core dumped)
In general, you can get TensorFlow+python3.6.9 installed with the instructions below:
For a certain NumPy version, you may meet an “illegal instructions” error.
This can be fixed via exporting an environment parameter:
$ export OPENBLAS_CORETYPE=ARMV8
If a python3.9 TensorFlow package is required, you can also build it from the source.
Below is an example from our user for your reference:
(please change the GPU architecture into sm_72 for Xavier)