Running ai docker containers on jetson orin nano with gpu support

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
,
9

Hi,

A possible reason is that the Ollama doesn’t build with GPU support.

It is decided dynamically so please check if the CUDA library can be found successfully when building time.
For example, we can build Ollama with GPU with the below command:

# Ollama
ENV LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:$LD_LIBRARY_PATH
ENV PATH=/usr/local/cuda-12.8/bin:$PATH
RUN git clone https://github.com/ollama/ollama && cd ollama
WORKDIR /opt/ollama
RUN cmake -DCMAKE_CUDA_ARCHITECTURES=87 -B build && cmake --build build

Thanks.