Llama-3.1-Nemotron-70B-Instruct An error occurred in MPI_Init_thread

AI & Data Science NVIDIA NIM Models
,
1

Download Llama-3.1-Nemotron-70B-Instructat from NIM
Using the following environment in nvidia_entrypoint , still getting the MPI error, what I missed?

LD_LIBRARY_PATH=/opt/hpcx/ucc/lib/ucc:/opt/hpcx/ucc/lib:/opt/hpcx/ucx/lib/ucx:/opt/hpcx/ucx/lib:/opt/hpcx/ompi/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/opt/nim/llm/.venv/lib/python3.10/site-packages/tensorrt_llm/libs:/opt/nim/llm/.venv/lib/python3.10/site-packages/nvidia/cublas/lib:/opt/nim/llm/.venv/lib/python3.10/site-packages/tensorrt_libs:/opt/nim/llm/.venv/lib/python3.10/site-packages/nvidia/nccl/lib

PATH=/opt/nim/llm/.venv/bin:/opt/hpcx/ucc/bin:/opt/hpcx/ucx/bin:/opt/hpcx/ompi/bin:/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin

OPAL_PREFIX=/opt/hpcx/ompi

HPCX_HOME=/opt/hpcx

which mpirun
/opt/hpcx/ompi/bin/mpirun

2

Hi @JadeLu the team is taking a look into this, and we will get back to you when we have a better idea of what’s happening. Thanks for your patience.