Cuda toolkit version problem when trying to run a python script on gpu through numba's jit & cuda modules on agx xavier

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi everyone,

I’m facing a lot of problem trying to run a pyhton script on my NVIDIA AGX XAVIER GPU using numba jit and cuda.
After resolving many errors, I’m feel I’m almost done with it ! But I get the following error :

File “/usr/lib/python3/dist-packages/numba/cuda/dispatcher.py”, line 42, in call
return self.compiled(*args, **kws)
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 701, in call
kernel = self.specialize(*args)
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 712, in specialize
kernel = self.compile(argtypes)
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 730, in compile
kernel.bind()
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 489, in bind
self._func.get()
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 370, in get
ptx = self.ptx.get()
File “/usr/lib/python3/dist-packages/numba/cuda/compiler.py”, line 342, in get
**self._extra_options)
File “/usr/lib/python3/dist-packages/numba/cuda/cudadrv/nvvm.py”, line 464, in llvm_to_ptx
libdevice = LibDevice(arch=opts.get(‘arch’, ‘compute_20’))
File “/usr/lib/python3/dist-packages/numba/cuda/cudadrv/nvvm.py”, line 340, in init
raise RuntimeError(MISSING_LIBDEVICE_FILE_MSG.format(arch=arch))
RuntimeError: Missing libdevice file for compute_50.
Please ensure you have package cudatoolkit 7.5.
Install package by:

conda install cudatoolkit=7.5

However, my Cuda compilation tools version is already release 10.2, V10.2.89.
Moreover, I red that conda is not supported buy agx xavier architecture

Should I try to install Anaconda ? and then install the cuda toolkit through conda ?
Should I downgrade my cudatoolkit ?
Is it possible to have 2 versions of the cuda toolkit running in the same machine ?

Thank you

3

Hi,

For Xavier, numba need to be compiled with sm_72 and CUDA toolkit 10.2.
For more detail, please check this tutorial for information:

Thanks.

4

Thx for this answer.
I already have tried this tutorial.
I tried twice after reflashing my card (installing llvm9 instead of llvm7 because version was too old)
Now the error I get is when installing numba with pip3 install numba

“TBB version is too old, 2019 update 5, i.e. TBB_INTERFACE_VERSION >= 11005 required” is displayed

But I already have the latest tbb version on my card which is 2017~U7-8
I guess version 11005 (2019 Update5) is not compatible

5

The workaround is to either build numba wheel inside a container, because tbb.h header won’t be found there, and numba won’t try to build with TBB.

Or you can temporarily move your tbb.h before compiling numba:

sudo mv /usr/include/tbb/tbb.h /usr/include/tbb/tbb.bak

You can then move it back after numba is installed.

1 Like