PyTorch for Jetson

Robotics & Edge Computing Jetson Systems Announcements
, ,
1325

Does the prebuilt PyTorch 1.10 of Jetson Nano support quantization engines? I met a problem like this
RuntimeError: Didn't find engine for operation quantized::conv2d_prepack NoQEngine
And when I check the availability of engine, I got
image

1326

@user29974 it was built with USE_QNNPACK=0 USE_PYTORCH_QNNPACK=0 because at some point I encountered errors building with QNNPack enabled.

1327

I am sorry but I am not sure how to apply these two commands USE_QNNPACK=0 USE_PYTORCH_QNNPACK=0. In fact I was running a .py file to test my quantized model. Should I use it like this python3 test.py USE_QNNPACK=0 USE_PYTORCH_QNNPACK=0? I did so and it returned RuntimeError: quantized engine QNNPACK is not supported. Thank you.

1328

You would need to rebuild & reinstall the PyTorch wheel (see the Build from Source section at the top), but with QNNPack/ect enabled.

1329

Got it, thank you.

1330

I am trying to run this on my jetson nano, jetpack 4, python 3.6.
When running the “python3 setup.py install --user” it fails with “illegal instruction (core dumped)”.
Any idea?

1331

Hi @byigal, my bet is it’s this numpy issue:

Try running export OPENBLAS_CORETYPE=ARMV8 in your terminal first.

1332

Thanks for the fast response!
The export command you suggested solved this error, yet then it failed later on: " error: ‘AutoDispatchBelowADInplaceOrView’ is not a member of ‘at’
at::AutoDispatchBelowADInplaceOrView g;"

I am trying to build an application with Jetson Nano to identify a toy train with coordinates from camera(s), and send messages to ESP32 that controls the train.
Am I going in the right direction? I feel that there is a lot of pain with Python’s packages and versions.

Also, this run is within venv, that is not importing the system packages. I don’t really need the venv because my Jetson Nano will run one application only, yet with the number of trials I do, I think I need to isolate the different runs until one will actually work for me. So far no luck :(

So far I could identify both cameras (CSI & USC) and see the video stream, yet no success with the recognition flow.

Is there a one simple to follow guide?

1333

@byigal I haven’t seen that error before, but presumably you are installing torchvision? If so, perhaps the version of torchvision you are trying to setup isn’t compatible with the version of PyTorch that you have installed.

Personally I use containers instead of venv, as they also capture apt/deb packages and other environment changes and I find have less issues than venv. If you still have problems installing PyTorch/torchvision, you can use the pre-built l4t-pytorch container from here: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch

1334

I was very focused on the guide lines, but it doesn’t mean I didn’t do a mistake. I will double check that.

I understand the rational behind the dockers, yet I never used them so far, so I need to learn. Does it mean with such a docker I have pre-installed all the flow I need in order to run custom model (I have it trained already) on the GPUs of my Jetson Nano? Is there any guide/video you would recommend me to watch to get into dockers on my Jetson Nano?

1335

@byigal the l4t-pytorch container already has PyTorch and torchvision pre-installed, and you can install other apt/pip libraries into it as needed (or perform any other configuration of the container’s environment that you need to). You can mount your files/models into the container and access them that way.

If you are running JetPack 4.6.1 or newer on your Nano, you should be able to run this to start the container:

sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-pytorch:r32.7.1-pth1.9-py3

or with your directory of choice mounted in:

sudo docker run -it --rm --runtime nvidia --network host -v /home/user/project:/location/in/container nvcr.io/nvidia/l4t-pytorch:r32.7.1-pth1.9-py3

If you’re running a version of JetPack older than JetPack 4.6.1 / L4T R32.7.1, there are other versions of the container listed here: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch

1336

hey @dusty_nv im stuck with installation of CUDA enabled torch compilation and torchvision installation for days now i followed above procedure and installed it and verified it cuda also showed true see the image below. it was perfect but then i tried passing device=0 in my yolov8 model to access gpu but torch.available_device returned zero and cuda showed false

258127212-c31caf98-647a-47b2-af36-512a87f483dc
258127212-c31caf98-647a-47b2-af36-512a87f483dc1600×900 179 KB

i believe we are not exporting PATH or LD_LIBRARY_PATH while following the above steps and hence we are facing this issue also maybe other errors which i dont know
please can you provide me with instruction to install
also when we are installing the above whl file are these files compiled such that CUDA is enabled
because i need CUDA enable torch and torchvision for yolov8. can you please provide me a step by step instructions im starting out and stuck with this problem for days
im trying to install torch compiled with CUDA enabled
my specs
JETSON NANO
jetpack v4.6.1
CUDA 10.2
i need
torch 1.10.0
torchvision 0.11.0
please and thanks alot

1337

@yogeshedigem2409 in your screenshot that Ultralytics says Python 3.8 and PyTorch 2.0.1, but you’re on Nano with PyTorch 1.10 and Python 3.6. I think that Ultralytics code is running under a different Python.

You could also try these for running YOLO with TensorRT:

1338

Hello.I couldnt install pytorch and asked question: Cant install Pytorch on JetsonNano P3450. Can i duplicate it here?

1339

Hello,

Anyone has tried to use PyTorch Ignite on a Jetson board ? I got following error when I try to load it:

python3 -c 'import ignite'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/ignite/__init__.py", line 3, in <module>
    import ignite.engine
  File "/usr/local/lib/python3.8/dist-packages/ignite/engine/__init__.py", line 10, in <module>
    from ignite.metrics import Metric
  File "/usr/local/lib/python3.8/dist-packages/ignite/metrics/__init__.py", line 7, in <module>
    from ignite.metrics.frequency import Frequency
  File "/usr/local/lib/python3.8/dist-packages/ignite/metrics/frequency.py", line 7, in <module>
    from ignite.handlers.timing import Timer
  File "/usr/local/lib/python3.8/dist-packages/ignite/handlers/__init__.py", line 5, in <module>
    from ignite.handlers.checkpoint import Checkpoint, DiskSaver, ModelCheckpoint
  File "/usr/local/lib/python3.8/dist-packages/ignite/handlers/checkpoint.py", line 17, in <module>
    from torch.distributed.optim import ZeroRedundancyOptimizer
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/optim/__init__.py", line 20, in <module>
    from .named_optimizer import _NamedOptimizer
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/optim/named_optimizer.py", line 10, in <module>
    from torch.distributed._shard.sharded_tensor import ShardedTensor
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/_shard/__init__.py", line 1, in <module>
    from .api import (
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/_shard/api.py", line 5, in <module>
    from torch.distributed import distributed_c10d
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 16, in <module>
    from torch._C._distributed_c10d import (
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package

I think it’s because torch has been built with option USE_DISTRIBUTED=0 and it is missing c10. Could you confirm my hypothesis ?

1340

hey @dusty_nv thanks alot i was able to install the pytorch and make the model run on torch1.11 in a python 3.8 virtual environment but i faced an issue while trying to accelerate the model using tensorrt .tensorrt is present locally in /usr/src/tensorrt but im not able to use it in virtual environment and got an error saying nvidia tensorrt moudule not found plus i dont remember downloading tensorrt locally should have come with jetpack how do i use tensorrt in a virtual environment? thanks again!!

1341

@jerome.haefliger yes I believe you are correct, if you encounter errors like that it’s typically because the Jetson builds of PyTorch are without distributed enabled. Sometimes you can just disable/comment out the offending code, or rebuild PyTorch with USE_DISTRIBUTED=1

1342

@yogeshedigem2409 are you able to import tensorrt through Python 3.6? If you are on Jetson Nano / JetPack 4, the Python bindings for TensorRT would have been installed for Python 3.6, not Python 3.8 (you can attempt to rebuild them for Python 3.8 from here). Otherwise if you are on JetPack 5, make sure your virtual environment is using the system site packages.

1343

Hi @A98, I’ve replied to your topic in the other thread - hope that helps.

1344

Building PyTorch on my Jetson Xavier NX is not so easy because I am running out of memory during the compilation and it gets killed by the kernel.

Do you have the chance to build PyTorch 2.0.1 with the option USE_DISTRIBUTED=1 ?