There appears to be a BIG problem with the 1.6.0 PyTorch wheel - it installs the wrong version (1.4.0!) This was driving me nuts until I read the output after installation carefully:
$ pip3 install numpy torch-1.6.0-cp36-cp36m-linux_aarch64.whl
Collecting numpy
Processing ./torch-1.6.0-cp36-cp36m-linux_aarch64.whl
Installing collected packages: numpy, torch
Successfully installed numpy-1.19.2 torch-1.4.0
Notice how, even though I downloaded and installed the “1.6.0” wheel, pip reports that I have installed torch-1.4.0.
I have resorted to using the “release candidate” version of this wheel (PyTorch for JetPack 4.4 - L4T R32.4.3 in Jetson Xavier NX) which worked flawlessly. (I am running JetPack 4.4 on a Xavier NX)