rsc44
May 4, 2022, 10:34pm
1
Hello,
I’m wondering if anyone, or if someone @2024a can share the complete steps for building CuDF and Cuspatial on any of the Jetson Devices.
I have tried to install these libraries many times, without any luck.
There are no guides available that actually work, i have tried the following guides/repos/gists. They do not work in 2022, on the latest l4t.
Required packages for cuDF, cuML, and cuGraph ARM support - GitHub - awthomp/rapids-jetson: Required packages for cuDF, cuML, and cuGraph ARM support
opened 06:22AM - 11 Feb 20 UTC
closed 02:37AM - 02 Jun 20 UTC
bug
cuDF (Python)
Cython
After a long struggle I have managed to install successfully libcudf, nvstrings … and dependencies such as dlpack, rmm on Jetson Nano (ARM - AArch64).
I had completely functional libarrow and Pyarrow already in my machine - [Pyarrow on ARM](https://gist.github.com/heavyinfo/04e1326bb9bed9cecb19c2d603c8d521).
But, I've reached a dead end when building cudf python bindings due to a error wehn compiling cython arrow files.
**Describe the bug**
```
sudo -E python3 setup.py build_ext --inplace
Compiling cudf/_lib/arrow/_cuda.pyx because it changed.
Compiling cudf/_lib/avro.pyx because it changed.
Compiling cudf/_lib/binops.pyx because it changed.
Compiling cudf/_lib/concat.pyx because it changed.
Compiling cudf/_lib/copying.pyx because it changed.
Compiling cudf/_lib/csv.pyx because it changed.
Compiling cudf/_lib/cudf.pyx because it changed.
Compiling cudf/_lib/dlpack.pyx because it changed.
Compiling cudf/_lib/filling.pyx because it changed.
Compiling cudf/_lib/gpuarrow.pyx because it changed.
Compiling cudf/_lib/groupby.pyx because it changed.
Compiling cudf/_lib/hash.pyx because it changed.
Compiling cudf/_lib/issorted.pyx because it changed.
Compiling cudf/_lib/join.pyx because it changed.
Compiling cudf/_lib/json.pyx because it changed.
Compiling cudf/_lib/nvtx.pyx because it changed.
Compiling cudf/_lib/orc.pyx because it changed.
Compiling cudf/_lib/parquet.pyx because it changed.
Compiling cudf/_lib/quantile.pyx because it changed.
Compiling cudf/_lib/reduce.pyx because it changed.
Compiling cudf/_lib/replace.pyx because it changed.
Compiling cudf/_lib/reshape.pyx because it changed.
Compiling cudf/_lib/rolling.pyx because it changed.
Compiling cudf/_lib/search.pyx because it changed.
Compiling cudf/_lib/sort.pyx because it changed.
Compiling cudf/_lib/stream_compaction.pyx because it changed.
Compiling cudf/_lib/table.pyx because it changed.
Compiling cudf/_lib/transpose.pyx because it changed.
Compiling cudf/_lib/typecast.pyx because it changed.
Compiling cudf/_lib/unaryops.pyx because it changed.
Compiling cudf/_lib/utils.pyx because it changed.
Compiling cudf/_libxx/column.pyx because it changed.
Compiling cudf/_libxx/copying.pyx because it changed.
Compiling cudf/_libxx/stream_compaction.pyx because it changed.
Compiling cudf/_libxx/table.pyx because it changed.
[ 1/35] Cythonizing cudf/_lib/arrow/_cuda.pyx
Error compiling Cython file:
------------------------------------------------------------
...
"""
def __cinit__(self, CudaBuffer obj):
self.buffer = obj
self.reader = new CCudaBufferReader(self.buffer.buffer)
self.set_random_access_file(
shared_ptr[RandomAccessFile](self.reader))
^
------------------------------------------------------------
cudf/_lib/arrow/_cuda.pyx:740:23: unknown type in template argument
Error compiling Cython file:
------------------------------------------------------------
...
buffering.
"""
def __cinit__(self, CudaBuffer buffer):
self.buffer = buffer
self.writer = new CCudaBufferWriter(self.buffer.cuda_buffer)
self.set_output_stream(shared_ptr[OutputStream](self.writer))
^
------------------------------------------------------------
cudf/_lib/arrow/_cuda.pyx:787:42: unknown type in template argument
Traceback (most recent call last):
File "setup.py", line 76, in <module>
ext_modules=cythonize(extensions),
File "/home/abishek/.local/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1096, in cythonize
cythonize_one(*args)
File "/home/abishek/.local/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1219, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: cudf/_lib/arrow/_cuda.pyx
```
**Steps/Code to reproduce bug**
Build cudf python bindings on ARM after building and installing libcudf.
**Expected behavior**
Successful compilation of cudf python bindings.
**Environment overview (please complete the following information)**
- Environment location: Bare-metal.
- Method of cuDF install: from source.
**Environment details**
<details><summary>Click here to see environment details</summary><pre>
**git***
commit d6b4794b4a3ed7e6577042596a32732bd62fd074 (HEAD -> branch-0.13, origin/branch-0.13, origin/HEAD)
Merge: ecfd1307a a0c64d997
Author: Raza Jafri <razajafri@users.noreply.github.com>
Date: Sat Feb 8 21:39:01 2020 -0800
Merge pull request #4069 from razajafri/stringcast
Added cast of numeric columns to/from string
**git submodules***
b165e1fb11eeea64ccf95053e40f2424312599cc thirdparty/cub (v1.7.1)
bcd545071c7a5ddb28cb6576afc6399eb1286c43 thirdparty/jitify (heads/cudf)
cdcda484d0c7db114ea29c3b33429de5756ecfd8 thirdparty/libcudacxx (0.8.1-99-gcdcda48)
a97a7380c76346c22bb67b93695bed19592afad2 thirdparty/libcudacxx/libcxx (heads/rapidsai-interop)
***OS Information***
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
Linux Titan 4.9.140-tegra #1 SMP PREEMPT Sat Oct 19 15:54:06 PDT 2019 aarch64 aarch64 aarch64 GNU/Linux
***GPU Information***
./print_env.sh: line 25: nvidia-smi: command not found
***CPU***
Architecture: aarch64
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Vendor ID: ARM
Model: 1
Model name: Cortex-A57
Stepping: r1p1
CPU max MHz: 1428.0000
CPU min MHz: 102.0000
BogoMIPS: 38.40
L1d cache: 32K
L1i cache: 48K
L2 cache: 2048K
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32
***CMake***
/usr/local/bin/cmake
cmake version 3.16.4
CMake suite maintained and supported by Kitware (kitware.com/cmake).
***g++***
/usr/bin/g++
g++ (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
***nvcc***
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326
***Python***
/usr/bin/python
Python 2.7.15
***Environment Variables***
PATH : /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/abishek/.local/bin:/usr/local/cuda/bin:/usr/local/cuda/bin
LD_LIBRARY_PATH : /usr/local/lib::/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
NUMBAPRO_NVVM :
NUMBAPRO_LIBDEVICE :
CONDA_PREFIX :
PYTHON_PATH :
conda not found
pip not found
</pre></details>
**Additional context**
Although env reports default Python as 2.7.15, I've used Python3 throughout my build.
If this bug is resolved, I expect a successful cudf installation on ARM.
Required packages for cuDF, cuML, and cuGraph ARM support - GitHub - JasonAtNvidia/rapids-jetson: Required packages for cuDF, cuML, and cuGraph ARM support
Any help, or pointers to get these libraries installed would be much appreciated.
Thank you.
Hi @rsc44 , RAPIDS isn’t supported on Jetson, sorry about that. RAPIDS currently requires CUDA 11.5 for building the latest libraries, while JetPack 5.0 is on CUDA 11.4. Official support for Jetson is something we are looking into a roadmap for, but can’t provide a definitive commitment or timeline currently.
rsc44
May 7, 2022, 9:46pm
6
Hello @dusty_nv ,
I am aware that the latest branch of RAPIDS is not supported to CUDA 11.5 being required, i shoud have made that more clear.
I am not able to properly install the branch of CUDF “branch-0.19” that supports CUDA 10.1+.
Are you saying that branch will also not work on the latest jetpack/l4t (5.0) ?
I re-flashed one of my Xaviers to jetpack 4.5 so that i can try installing CuDF again.
I’ll log my steps, and let you know if i’m successful.
I’m not sure if it will work or not as it’s not officially supported on Jetson, but let us know if you are able to get it working with a bit of hacking, and I’m sure the community will appreciate it. Sorry to not be of more direct assistance as I’m not personally experienced with RAPIDS. You could try logging an issue on that GitHub if that helps.
system
Closed
June 1, 2022, 2:23am
9
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.