USE_QNNPACK=0 and USE_PYTORCH_QNNPACK=0

I was hoping to get any rationale for why USE_QNNPACK and USE_PYTORCH_QNNPACK are set to 0 when building pytorch for the jetpack (build from source section of PyTorch for Jetson and also verified while using it). Is this an architectural limitation? I don’t want to go down the building from source route if there is some inherent reason QNNPACK won’t work.

Thanks!

Hi,

GitHub - pytorch/QNNPACK: Quantized Neural Network PACKage - mobile-optimized implementation of quantized neural network operators

QNNPACK (Quantized Neural Networks PACKage) is a mobile-optimized library for low-precision high-performance neural network inference. QNNPACK provides implementation of common neural network operators on quantized 8-bit tensors.

Based on the description. QNNPACK targets for low-precision and high-performance inference.
This is exactly what TensorRT can do. Please try to convert the model into a TensorRT engine instead.

If you prefer to use QNNPACK, please build PyTorch from the source on your own.
We don’t verify if QNNPACK works on Jetson but it seems to work on ARM system based on their document.

Thanks.

@nbnv at some point I was encountering build errors associated with QNNPACK, so I disabled it. If you need it, you can try enabling it again to see if it builds now with more recent versions of PyTorch.

Enabling QNNPACK and re-building works. Execution so far looks good. I am on 35.2.1 and using pytorch 2.0.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.