@tokada1 yes, you need to set LD_LIBRARY_PATH=/usr/lib/cuda/lib64
(or include that directory in your library path if you already have it set) because the bundled libraries don’t seem to work on Jetson devices, we haven’t quite worked that problem out yet.
I am getting same error and my LD_LIBRARY_PATH is already set. To /usr/local/cuda-12.2/lib64
yeah, I had to build ollama from source to make it work on JP6 GA system. I guess you need to build it for the specific CUDA (major?) version on the system?
@remy415 have you tried it on JP6 GA? If it works for you, can you comment here?:
@tokada1 if I build it on JP6, it works. There’s some kind of incompatibility with JP6 that hasn’t been worked out on the Ollama side. I haven’t had much time to poke around and see what the issue is, I know dhiltgen on the Ollama git has been working on it too. If you find anything please let me know.
@tokada1 I rebuilt ollama container for R36.3: dustynv/ollama:r36.3.0
Other R36.2 container images that I tried on R36.3 were compatible (I did not try this one yet)
Thanks for the new image. :)
IIRC, R36.2 image worked fine on the JP6 GA system.
I 'm sure you know this, but just to be clear, the problem I have is when I run Ollama on the host directly, not using docker.
The pre-built binary is built against CUDA 11, but I thought it should work with CUDA 12 driver as long as the binary includes the v11 runtime.
Anyway, this is not a jetson-container issue. ;)
I found that 35.4.1 works fine on my Jetson Xavier NX. I was unable to get 36.2.0 working as it seems to require pytorch 2.3. I couldn’t find a way to get that one installed.
trying to install the container on a jetson tx1 (not sure this container runs on that hardware or not), but it fails in install.sh as follows: FYI.
+++ readlink -f jetson-containers/install.sh
++ dirname /home/nvidia/workspace/test/jetson-containers/install.sh
-
ROOT=/home/nvidia/workspace/test/jetson-containers
-
INSTALL_PREFIX=/usr/local/bin
-
pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6) -
pip3 install -r /home/nvidia/workspace/test/jetson-containers/requirements.txt
Collecting git+https://github.com/Granulate/DockerHub-API.git (from -r /home/nvidia/workspace/test/jetson-containers/
requirements.txt (line 4))
Cloning GitHub - Granulate/DockerHub-API: Python Docker Hub API client library to /tmp/pip-dvn3j4yi-build
Collecting packaging>=20.0 (from -r /home/nvidia/workspace/test/jetson-containers/requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572
d127/packaging-21.3-py3-none-any.whl
Collecting pyyaml>=6 (from -r /home/nvidia/workspace/test/jetson-containers/requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/cd/e5/af35f7ea75cf72f2cd079c95ee16797de7cd71f29ea7c68ae5ce7be1eda0/PyYAML-6.0.1.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/PyYAML.egg-info
writing pip-egg-info/PyYAML.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/PyYAML.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/PyYAML.egg-info/top_level.txt
writing manifest file ‘pip-egg-info/PyYAML.egg-info/SOURCES.txt’
Traceback (most recent call last):
File “”, line 1, in
File “/tmp/pip-build-2kdhg8am/pyyaml/setup.py”, line 313, in
python_requires=‘>=3.6’,
File “/usr/lib/python3/dist-packages/setuptools/init.py”, line 129, in setup
return distutils.core.setup(**attrs)
File “/usr/lib/python3.6/distutils/core.py”, line 148, in setup
dist.run_commands()
File “/usr/lib/python3.6/distutils/dist.py”, line 955, in run_commands
self.run_command(cmd)
File “/usr/lib/python3.6/distutils/dist.py”, line 974, in run_command
cmd_obj.run()
File “/usr/lib/python3/dist-packages/setuptools/command/egg_info.py”, line 278, in run
self.find_sources()
File “/usr/lib/python3/dist-packages/setuptools/command/egg_info.py”, line 293, in find_sources
mm.run()
File “/usr/lib/python3/dist-packages/setuptools/command/egg_info.py”, line 524, in run
self.add_defaults()
File “/usr/lib/python3/dist-packages/setuptools/command/egg_info.py”, line 560, in add_defaults
sdist.add_defaults(self)
File “/usr/lib/python3/dist-packages/setuptools/command/py36compat.py”, line 36, in add_defaults
self._add_defaults_ext()
File “/usr/lib/python3/dist-packages/setuptools/command/py36compat.py”, line 120, in _add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
File “/tmp/pip-build-2kdhg8am/pyyaml/setup.py”, line 205, in get_source_files
self.cython_sources(ext.sources, ext)
File “/usr/lib/python3.6/distutils/cmd.py”, line 103, in getattr
raise AttributeError(attr)
AttributeError: cython_sources
Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-build-2kdhg8am/pyyaml/
@jspatel this is an issue with pyaml as discussed here:
As to how to fix it, I am unsure. Possibly changing your requirements.txt somehow to force a working version of the library. You may need to wait for the repo manager to update the file(s). Check the referenced link for more information.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.