AGX JetPack 4.6: install tensorflow and rasterio with conda

All,
I’m attempting to create a conda (miniconda) environment with tensorflow and rasterio. I can get rasterio to install in a conda environment but not the base python (/usr/bin/python3) and I am unable to get tensorflow to install inside a conda environment. I have tried lots of things, and the sudo pip3 install instructions found here: How to Install JetPack :: NVIDIA JetPack Documentation inform me that it is already satisfied, but when I launch python 3.9 from the conda enviro, it is not installed and will not import.

I’m at my wits end. I’ve gotten the system up and running with very few problems (a few) but can’t sort out how to get this going. I don’t want to flash and start all over if I don’t have to, but now tensorflow will not import in the 3.6 /usr/bin/python3 python either… Illegal Instruction core dump error, so presumeably I broke something trying to get it to install tf into the conda environment.

Any help appreciated. What can I output to help?

Thanks,
seth

SOLVED

2 things. I needed to downgrade python to 3.6.

And then there was pip3…
It turns out that I was overlooking the fact that when I call pip3 as sudo pip3, it is not calling the anaconda installed pip3. This is how I sorted it out:

(geo) <user>@agx:~$ which -a pip
/home/<user>/miniforge3/envs/geo/bin/pip
/usr/local/bin/pip

(geo) <user>@agx:~$ which -a pip3
/home/<user>/miniforge3/envs/geo/bin/pip3
/usr/local/bin/pip3
/usr/bin/pip3

(geo) <user>@agx:~$ which pip3
/home/<user>/miniforge3/envs/geo/bin/pip3

(geo) <user>@agx:~$ sudo which pip3
[sudo] password for <user>:
/usr/local/bin/pip3

(geo) <user>@agx:~$ sudo /home/<user>/miniforge3/envs/geo/bin/pip3 install --pre --no-cache-dir --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow

Looking in indexes: https://pypi.org/simple, https://developer.download.nvidia.com/compute/redist/jp/v46
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow

(geo) <user>@agx:~$ conda install python=3.6
...
(geo) <user>@agx:~$ sudo /home/<user>/miniforge3/envs/geo/bin/pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow

I included the full path to the desired pip3 (anaconda) and it seems to be installed now.

But now there is a new problem:

(geo) seth@agx:~$ python
Python 3.6.13 | packaged by conda-forge | (default, Feb 19 2021, 05:46:38)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2021-08-26 13:43:20.500471: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.10.2
/home/seth/miniforge3/envs/geo/lib/python3.6/site-packages/h5py/__init__.py:40: UserWarning: h5py is running against HDF5 1.10.6 when it was built against 1.10.0, this may cause problems
  '{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.0, library is 1.10.6
	    SUMMARY OF THE HDF5 CONFIGURATION
	    =================================

General Information:
-------------------
                   HDF5 Version: 1.10.6
                  Configured on: Sat Jan  9 02:03:46 UTC 2021
                  Configured by: conda@2af0a685c269
                    Host system: aarch64-conda-linux-gnu
              Uname information: Linux 2af0a685c269 4.15.0-36-generic #39~16.04.1-Ubuntu SMP Tue Sep 25 09:00:05 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
                       Byte sex: little-endian
             Installation point: /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh

Compiling Options:
------------------
                     Build Mode: production
              Debugging Symbols: no
                        Asserts: no
                      Profiling: no
             Optimization Level: high

Linking Options:
----------------
                      Libraries: static, shared
  Statically Linked Executables:
                        LDFLAGS: -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib -Wl,-rpath-link,/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib -L/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib
                     H5_LDFLAGS:
                     AM_LDFLAGS:  -L/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib
                Extra libraries: -lcrypto -lcurl -lpthread -lz -ldl -lm
                       Archiver: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-ar
                       AR_FLAGS: cr
                         Ranlib: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-ranlib

Languages:
----------
                              C: yes
                     C Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-cc
                       CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include
                    H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L   -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:  -I/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include
                        C Flags: -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/work=/usr/local/src/conda/hdf5_split-1.10.6 -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix
                     H5 C Flags:  -std=c99  -pedantic -Wall -Wextra -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual -Wconversion -Wdeclaration-after-statement -Wdisabled-optimization -Wfloat-equal -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-enum -Wswitch-default -Wundef -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings -Wlogical-op -Wlarger-than=2560 -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wnull-dereference -Wunused-const-variable -Wduplicated-cond -Whsa -Wnormalized -Walloc-zero -Walloca -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -fstdarg-opt -s -Wno-inline -Wno-aggregate-return -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-suggest-attribute=const -Wno-suggest-attribute=pure -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
                     AM C Flags:
               Shared C Library: yes
               Static C Library: yes


                        Fortran: yes
               Fortran Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-gfortran
                  Fortran Flags:
               H5 Fortran Flags:  -pedantic -Wall -Wextra -Wunderflow -Wimplicit-interface -Wsurprising -Wno-c-binding-type  -s -O2
               AM Fortran Flags:
         Shared Fortran Library: yes
         Static Fortran Library: yes

                            C++: yes
                   C++ Compiler: /drone/src/build_artifacts/hdf5_split_1610157654531/_build_env/bin/aarch64-conda-linux-gnu-c++
                      C++ Flags: -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/work=/usr/local/src/conda/hdf5_split-1.10.6 -fdebug-prefix-map=/drone/src/build_artifacts/hdf5_split_1610157654531/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix
                   H5 C++ Flags:   -pedantic -Wall -W -Wundef -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor -Wctor-dtor-privacy -Wabi -finline-functions -s -O
                   AM C++ Flags:
             Shared C++ Library: yes
             Static C++ Library: yes

                           Java: no


Features:
---------
                   Parallel HDF5: no
Parallel Filtered Dataset Writes: no
              Large Parallel I/O: no
              High-level library: yes
                Build HDF5 Tests: yes
                Build HDF5 Tools: yes
                    Threadsafety: yes
             Default API mapping: v110
  With deprecated public symbols: yes
          I/O filters (external): deflate(zlib)
                             MPE: no
                      Direct VFD: no
              (Read-Only) S3 VFD: yes
            (Read-Only) HDFS VFD: no
                         dmalloc: no
  Packages w/ extra debug output: none
                     API tracing: no
            Using memory checker: yes
 Memory allocation sanity checks: no
          Function stack tracing: no
       Strict file format checks: no
    Optimization instrumentation: no
Bye...
Aborted (core dumped)

Any ideas how to solve this?

OK. Solved that:
conda uninstall hdf5
and suddenly tensorflow will import, but rasterio won’t install

:-(

Hi,

In general, you can get TensorFlow+python3.6.9 installed with the instructions below:
https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html#prereqs

For a certain NumPy version, you may meet an “illegal instructions” error.
This can be fixed via exporting an environment parameter:

$ export OPENBLAS_CORETYPE=ARMV8

If a python3.9 TensorFlow package is required, you can also build it from the source.
Below is an example from our user for your reference:
(please change the GPU architecture into sm_72 for Xavier)

Thanks.