Jetpack with cuda-aware OMPI could be default

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
7

Hi,

We can build OpenMPI+CUDA on JetPack 4.6 without issues.
Below is our building steps for your reference:

1. Set environment

$ export CUDA_HOME="/usr/local/cuda"
$ export UCX_HOME="/usr/local/ucx"
$ export OMPI_HOME="/usr/local/ompi"

$ export PATH="${CUDA_HOME}/bin:$PATH}"
$ export PATH="{UCX_HOME}/bin:$PATH}"
$ export PATH="{OMPI_HOME}/bin:$PATH}"

$ export LD_LIBRARY_PATH="${CUDA_HOME}/lib64:$LD_LIBRARY_PATH}"
$ export LD_LIBRARY_PATH="${UCX_HOME}/lib64:$LD_LIBRARY_PATH}"
$ export LD_LIBRARY_PATH="${OMPI_HOME}/lib64:$LD_LIBRARY_PATH}"

2. Install UCX

$ git clone https://github.com/openucx/ucx
$ cd ucx/

$ cd ucx/
$ git clean -xfd
$ ./autogen.sh
$ mkdir build
$ cd build
$ ../configure --prefix=$UCX_HOME --enable-debug --with-cuda=$CUDA_HOME --enable-mt --disable-cma

$ make
$ sudo make install

3. Install MPI

$ git clone https://github.com/open-mpi/ompi.git
$ cd ompi/
$ git submodule update --init --recursive

$ sudo apt-get install -y pandoc
$ ./autogen.pl
$ mkdir build
$ cd build
$ ./configure --with-cuda=$CUDA_HOME --with-ucx=$UCX_HOME

$ make
$ sudo make install

4. Verified

$ ompi_info -a | grep "\-with\-cuda"
Configure command line: '--with-cuda=/usr/local/cuda' '--with-ucx=/usr/local/ucx'

Thanks.

1 Like