This was on Jetpack 4.4 but should still apply to anyone who finds themselves here.
h5py version 3 resolves the issue, it was not on pip yet but you can install from the git repo using the following.
git clone https://github.com/h5py/h5py.git
cd h5py/
pip3 install .
You may also be able to
pip3 install git+https://github.com/h5py/h5py.git