Failed building wheel for h5py on Jetpack 4.6.1 on Xavier NX

[cydu]

Hi, I encountered a problem when I tried to install Tensorflow to my fresh Jetpack 4.6.1 on Xavier NX. I followed the steps on this site: Installing TensorFlow for Jetson Platform :: NVIDIA Deep Learning Frameworks Documentation. All installations are OK until coming to this step: $ sudo env H5PY_SETUP_REQUIRES=0 pip3 install -U h5py==3.1.0. An error occurred in this step:
Building wheel for h5py (pyprlject.toml) … error
Failed to build h5py
How to solve this problem?
Can I proceed to install Tensorflow without h5py?

[AastaLLL]

Hi,

Yes, could you proceed to install the package to see if any issue?
Thanks.

[cydu]

Hi,
I tried, with 2 attempts:
sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v50 tensorflow (this is the original command)
and
sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow (after 1st failure, I tried this command because I’m now using Jetpack 4.6.1)
But both failed.
Please advise.

[cydu]

Hi,
Can you please reply? I’m waiting for your solution to this issue.

[AastaLLL]

Hi,

Could you share more details about the failure?
Do you get any error output can share with us?

More, we also have a container that has TensorFlow pre-installed.
Maybe you can use it directly for simplicity.

NVIDIA L4T TensorFlow | NVIDIA NGC

Thanks.

[cydu]

Hi,
Below are the errors when I use this command $ sudo env H5PY_SETUP_REQUIRES=0 pip3 install -U h5py==3.1.0 to install h5py.
Collecting h5py==3.1.0
Downloading h5py-3.1.0.tar.gz (371 kB)
|████████████████████████████████| 371 kB 324 kB/s
Installing build dependencies … done
Getting requirements to build wheel … done
Preparing metadata (pyproject.toml) … done
Requirement already satisfied: numpy>=1.12 in /usr/local/lib/python3.6/dist-packages (from h5py==3.1.0) (1.19.4)
Collecting cached-property
Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Building wheels for collected packages: h5py
Building wheel for h5py (pyproject.toml) … error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /home/amtd-avs/.local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/in_process.py build_wheel /tmp/tmpjtwlqha
cwd: /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60
Complete output (120 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.6
creating build/lib.linux-aarch64-3.6/h5py
copying h5py/h5py_warnings.py → build/lib.linux-aarch64-3.6/h5py
copying h5py/ipy_completer.py → build/lib.linux-aarch64-3.6/h5py
copying h5py/init.py → build/lib.linux-aarch64-3.6/h5py
copying h5py/version.py → build/lib.linux-aarch64-3.6/h5py
creating build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/attrs.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/files.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/selections2.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/base.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/datatype.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/compat.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/dataset.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/group.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/selections.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/filters.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/dims.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/init.py → build/lib.linux-aarch64-3.6/h5py/_hl
copying h5py/_hl/vds.py → build/lib.linux-aarch64-3.6/h5py/_hl
creating build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dtype.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5d_direct_chunk.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_attrs_data.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_file2.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_slicing.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/common.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5t.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_base.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_group.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_file_image.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dims_dimensionproxy.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5p.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dataset.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_completions.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dataset_swmr.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_attrs.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_selections.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_file.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5f.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/init.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_big_endian_file.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/conftest.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dimension_scales.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_datatype.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_objects.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_errors.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_h5pl.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_filters.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_attribute_create.py → build/lib.linux-aarch64-3.6/h5py/tests
copying h5py/tests/test_dataset_getitem.py → build/lib.linux-aarch64-3.6/h5py/tests
creating build/lib.linux-aarch64-3.6/h5py/tests/data_files
copying h5py/tests/data_files/init.py → build/lib.linux-aarch64-3.6/h5py/tests/data_files
creating build/lib.linux-aarch64-3.6/h5py/tests/test_vds
copying h5py/tests/test_vds/test_highlevel_vds.py → build/lib.linux-aarch64-3.6/h5py/tests/test_vds
copying h5py/tests/test_vds/test_virtual_source.py → build/lib.linux-aarch64-3.6/h5py/tests/test_vds
copying h5py/tests/test_vds/init.py → build/lib.linux-aarch64-3.6/h5py/tests/test_vds
copying h5py/tests/test_vds/test_lowlevel_vds.py → build/lib.linux-aarch64-3.6/h5py/tests/test_vds
copying h5py/tests/data_files/vlen_string_s390x.h5 → build/lib.linux-aarch64-3.6/h5py/tests/data_files
copying h5py/tests/data_files/vlen_string_dset.h5 → build/lib.linux-aarch64-3.6/h5py/tests/data_files
copying h5py/tests/data_files/vlen_string_dset_utc.h5 → build/lib.linux-aarch64-3.6/h5py/tests/data_files
running build_ext
Loading library to get version: /usr/lib/aarch64-linux-gnu/hdf5/serial/libhdf5.so

---

                     Summary of the h5py configuration

HDF5 include dirs: [
‘/usr/include/hdf5/serial’
]
HDF5 library dirs: [
‘/usr/lib/aarch64-linux-gnu/hdf5/serial’
]
HDF5 Version: (1, 10, 0)
MPI Enabled: False
Rebuild Required: True

---

Executing api_gen rebuild of defs
Executing cythonize()
[ 1/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/_conv.pyx
[ 2/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/_errors.pyx
[ 3/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/_objects.pyx
[ 4/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/_proxy.pyx
[ 5/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/_selector.pyx
[ 6/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/defs.pyx
[ 7/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5.pyx
[ 8/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5a.pyx
[ 9/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5ac.pyx
[10/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5d.pyx
[11/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5ds.pyx
[12/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5f.pyx
[13/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5fd.pyx
[14/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5g.pyx
[15/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5i.pyx
[16/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5l.pyx
[17/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5o.pyx
[18/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5p.pyx
[19/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5pl.pyx
[20/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5r.pyx
[21/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5s.pyx
[22/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5t.pyx
[23/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/h5z.pyx
[24/24] Cythonizing /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/utils.pyx
building ‘h5py.defs’ extension
creating build/temp.linux-aarch64-3.6
creating build/temp.linux-aarch64-3.6/tmp
creating build/temp.linux-aarch64-3.6/tmp/pip-install-876igjbp
creating build/temp.linux-aarch64-3.6/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60
creating build/temp.linux-aarch64-3.6/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DH5_USE_18_API -DNPY_NO_DEPRECATED_API=0 -I./h5py -I/usr/include/hdf5/serial -I/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/lzf -I/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/lzf/lzf -I/usr/local/lib/python3.6/dist-packages/numpy/core/include -I/usr/include/python3.6m -c /tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/defs.c -o build/temp.linux-aarch64-3.6/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/defs.o
/tmp/pip-install-876igjbp/h5py_1f4960f50a364da5a27aaab0ac28cc60/h5py/defs.c:54:10: fatal error: Python.h: No such file or directory
#include “Python.h”
^~~~~~~~~~
compilation terminated.
error: command ‘aarch64-linux-gnu-gcc’ failed with exit status 1

ERROR: Failed building wheel for h5py
Failed to build h5py
ERROR: Could not build wheels for h5py, which is required to install pyproject.toml-based projects

[cydu]

After above installation failure, I tried to install Tensorflow with this command : $ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow
I want to install Tensorflow on JP4.6.1. Below are the error messages:

WARNING: The directory ‘/home/amtd-avs/.cache/pip’ or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo’s -H flag.
Looking in indexes: Simple index, https://developer.download.nvidia.com/compute/redist/jp/v46
Collecting tensorflow
Downloading https://developer.download.nvidia.com/compute/redist/jp/v46/tensorflow/tensorflow-2.6.2%2Bnv21.12-cp36-cp36m-linux_aarch64.whl (317.1 MB)
|████████████████████████████████| 317.1 MB 2.2 MB/s
Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.37.1)
Requirement already satisfied: numpy~=1.19.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.19.4)
Collecting google-pasta~=0.2
Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 689 kB/s
Collecting keras<2.7,>=2.6.0
Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 1.7 MB/s
Collecting absl-py==0.12.0
Downloading absl_py-0.12.0-py3-none-any.whl (129 kB)
|████████████████████████████████| 129 kB 2.6 MB/s
Requirement already satisfied: gast==0.4.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.4.0)
Collecting opt-einsum~=3.3.0
Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
|████████████████████████████████| 65 kB 1.2 MB/s
Collecting clang~=5.0
Downloading clang-5.0.tar.gz (30 kB)
Preparing metadata (setup.py) … done
Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.1.2)
Collecting tensorboard<2.7,>=2.6.0
Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
|████████████████████████████████| 5.6 MB 1.8 MB/s
Collecting termcolor~=1.1.0
Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Preparing metadata (setup.py) … done
Collecting flatbuffers~=1.12.0
Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting wrapt~=1.12.1
Downloading wrapt-1.12.1.tar.gz (27 kB)
Preparing metadata (setup.py) … done
Requirement already satisfied: six~=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.15.0)
Requirement already satisfied: typing-extensions~=3.7.4 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (3.7.4.3)
Collecting tensorflow-estimator<2.7,>=2.6.0
Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
|████████████████████████████████| 462 kB 1.4 MB/s
Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (3.19.4)
Collecting astunparse~=1.6.3
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting grpcio<2.0,>=1.37.0
Downloading grpcio-1.46.1-cp36-cp36m-manylinux_2_17_aarch64.whl (60.2 MB)
|████████████████████████████████| 60.2 MB 5.0 MB/s
Collecting h5py~=3.1.0
Downloading h5py-3.1.0.tar.gz (371 kB)
|████████████████████████████████| 371 kB 3.3 MB/s
Installing build dependencies … done
Getting requirements to build wheel … done
Installing backend dependencies … error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /home/amtd-avs/.local/lib/python3.6/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-yi1_xslk/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i Simple index --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 --pre – ‘Cython>=0.29.14; python_version >= “3.8”’ ‘numpy==1.12; python_version == “3.6”’ ‘Cython>=0.29; python_version < “3.8”’ ‘numpy==1.14.5; python_version == “3.7”’ pkgconfig ‘numpy==1.17.5; python_version == “3.8”’ ‘numpy==1.19.3; python_version >= “3.9”’
cwd: None
Complete output (570 lines):
WARNING: The directory ‘/home/amtd-avs/.cache/pip’ or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo’s -H flag.
…
…many many lines of error messages after

[AastaLLL]

Hi,

Thanks for sharing the log.

Could you run the following command and try it again?

$ sudo apt-get install python3-dev

We can install the TensorFlow successfully on the JetPack 4.6.1 + NX with the instructions from the document (including the h5py installation).

Thanks.

[cydu]

Hi, Aasta:
Thank you.
I tried using
$ sudo apt-get install python3-dev
I installed TensorFlow successfully, but it brings me new problems. Now I cannot run my code. The system cannot import all packages, like PIL, CV2, numpy, tensorflow, etc., all with same error message “ModuleNotFoundError: No module named ***”. May I know if this command installs a new version (python3) of Python on the machine? How can i fix such problem?

[cydu]

I can run the code on my Xavier NX with Jetpack 4.5.1 before this upgrading.

[AastaLLL]

Hi,

It should just install some python header that requires for compiling the h5py library.
Do you run the packages with python2 or python3 before?

Thanks.

[cydu]

Hi Aasta:
No, I never ran my software before because I could not run it since I installed Jetpack 4.6.1.
Initially I ran Jetpack 4.5.1 on my Xavier NX. A few months ago, the user of my software said he can not run my software on Jetpack4.6.0 (because he can only download 4.6.0 from Nvidia website), So I upgrade my Xavier NX to Jetpack 4.6.0. Unfortunately I also cannot run the software. You gave me a temporary solution to solve this problem, and said you will fix this bug in next release. So I made a try of Jetpack 4.6.1 this month.

[cydu]

Hi Aasta:
I tried to re-download the image file of JetPack 4.6.1 and re-flash it to Xavier NX. Very fortunately, I succeeded this time. Don’t know the reason. Now I installed Tensorflow to the device. I can run my software in Pycharm. But I encountered the same problem as in JetPack 4.6.0. When I build the software to Executable with Pyinstaller, the exe cannot run (for details, please refer to this link JetPack 4.6 does not support Opencv when compiled by Pyinstaller - Jetson & Embedded Systems / Jetson Xavier NX - NVIDIA Developer Forums. You said this issue will be fixed in next release, namely v4.6.1, but the problem still exists. Can you please help?

[AastaLLL]

Hi,

Good to know the TensorFlow issue is solved now.

For the PyInstaller issue, the failure is caused by the different versions between PyInstaller and JetPack.
PyInstaller uses v4.5.4 but JetPack 4.6 use 4.1.1 and triggers the error.

Cannot use opencv on Xavier NX

Hi, I developed a vision inspection software on Xavier NX. This software uses OpenCV for image process. It works well on a few of Xavier NX devices which is installed with Jetpack 4.5. But when I deployed this software on a new Xavier NX recently, I encountered a problem. The new Xavier NX is installed with Jetpack 4.6 from your download web pages. Below is the screenshot of error message: [Err_Screenshot] Please take note that Line 10 is this command in the script: import cv2. This means I c…

However, since JetPack 4.6.1 is just a minor update, the OpenCV version remains 4.1.1.
To get OpenCV v4.5.4, please try our JetPack 5.0.1 DP below instead:

JetPack SDK

Thanks.

[cydu]

Hi Aasta:
I tried to use Jetpack 5.0.1 DP, but not successful. I downloaded the SD card image of Jetpack 5.0.1 DP, and flashed it to my 64GB SD card following normal procedures, but could not boot my Xavier NX with the card. The screen is black. I tried 2 times, but in vain.

[kayccc]

@cydu

I tried to use Jetpack 5.0.1 DP, but not successful.

Any log or screenhot as reference?

[cydu]

No, because I cannot boot the system. Totally nothing is shown on the monitor.

[cydu]

Hi,
Have you tested Xavier NX with Jetpack 5.0.1 DP? Can you install? What’s the problem?

[AastaLLL]

Hi,

Sorry that this is a known issue. You can find more information below:

After flashing the jetpack5.0.1 sd card image on the jetson devkit 8g ddr model and installing the sd card, it does not boot

Hello, After flashing the jetpack5.0.1 sd card image on the jetson devkit 8g ddr model and installing the sd card, it does not boot. file name: jetson-nx-jp501dp-sd-card-image.zip content in sdcard [image] [image] Thank you.

To upgrade from JetPack 4.x to JetPack 5.x, it’s required to use SDK manager to reflash for the first time.

Thanks.