My installation is as follows:
Jetpack 4.6.3 installed via the SD image
Tensorflow (not Tensorflow-GPU but just Tensorflow)
When I run the inferencing I get the following errors/messages:
“W tensorflow/stream_executor/platform/default/dso_loader.cc:65] Could not load dynamic library ‘libcudart.so.10.2’; dlerror: libcudart.so.10.2: cannot open shared object file: No such file or directory”
Then I get this message:
" : I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
TF Interpreter initialized …
Ready to accept new files … "
The program works but looks like it is doing it without the CUDA support.
How do I get the CUDA support?
The options I see are as follows:
Install Tensorflow-gpu instead of simple Tensorflow: Am in the process of doing it but I doubt it will work since CUDA installation should be part of the normal jetpack 4.6.3 install.
Install CUDA support using the command: sudo apt install nvidia-cuda
The reason why I am not opting for option 2 is when I do that it says it will fill up 2.99GB of my nano flash space, when all I need is just the runtime 'libcudart.so.10.2
I will be grateful for directions.
Do you have this directory? Or one with a different version number perhaps? /usr/local/cuda-10.2
If so, then do you have the subdirectory to this: /usr/local/cuda-10.2/aarch64/lib
The CUDA install is to the former, the library would exist in the latter. If you have the latter, then this contains the actual library. To tell the linker to look there make sure you have this file (you can create it if it is missing): /etc/ld.so.conf.d/cuda-10-2.conf
The content of that file:
So first you find the location of the actual .so file. It should be in the “/usr/local/cuda-<version>”. Then add the actual library directory of that installation (the “aarch64-linux/lib” subdirectory of that) full path to a file named after the cuda-<release> (uses hyphens for convention) in the linker’s extra search locations. Then reboot or run “sudo ldconfig”.
Thankyou for responding @linuxdev . The simple answer is no I dont have anything with the namecuda in /usr/local or /usr (* meaning anything before or after the text ‘cuda’). That is what worries me and surprises me . I searched the entire directory /usr for "cudart"and found only header files related to “cudaRt” deep into python3 sub directory, for example in:/usr/local/lib/python3.6/dist-packages/tensorflow/include/external/nvtx_archive/_virtual_includes/nvtx/third_party/nvtx3.
There are no binaries related to cuda in the install at all. This is when I did the install from jetpack 4.6.3. It is very clear to me that for some reason the cuda run-time is NOT getting installed from the jetpack.
I request help on this
One would normally use SDK Manager to install many of the optional packages (including CUDA). If you run SDKM using the release installed on your Jetson, you can uncheck things you don’t want, e.g., you could uncheck flash and uncheck installing things to the host PC. Then the host PC installs that content to a fully booted Jetson over ssh. If you have CUDA, but it isn’t in the “/usr/local/*cuda*” location, then likely you got an incompatible version for a discrete GPU instead of an integrated GPU.
Note that flashing from JetPack won’t install those optional libraries during the flash step. That install is only after a flash completes, first boot account setup is finished, and then the ssh access can start. The fact that you installed from JetPack 4.6.3 is good, but it sounds as if it flashed and never got to the step of installing CUDA.
Can you start your Jetson, fully boot it, and run JetPack again such that everything is unchecked except for installing CUDA? See what happens, and if it says there is an error, report that.
Thank you @linuxdev . I salute yr persistence and am grateful for your response.
Yes we did try with the SDK manager but it fails to recognize the board since the carrier board is made by another party. I find that very strange. You license the manufacturing of yr carrier board to someone and then dont include it in yr normal tools??? That is not done. When we contacted the supplier of the carrier board yesterday all he had to say is we need to move the OS to the SD card and then install the CUDA separately (yes all the 3Gig of it) in the internal drive. He also does not seem to know how to get the OS and CUDA together on the internal memory…something very basic as getting yr packages on a platform seems terribly broken. We will now have to experiment with moving the OS to a SD card (and the attended speed issues) and see if finally with CUDA in place (if at all it gets in) we get any performance improvement. Finally that is the goal. This is a good lesson of how a good product but TERRIBLE SUPPORT can derail development .
I’m not sure how third party installers work with that. The Jetson is not in recovery mode and uses ssh, so the usual USB ID system does not apply. One would normally enter an IP address manually in similar situations and use wired ethernet (or the IP address of Wi-Fi if that is set up). I can’t imagine that there isn’t some way for this to work, perhaps it is just a case of the third party manufacturer providing the “equivalent” installer step which allows unchecking flash and simply installing CUDA.
There is likely also a manual edit of the apt repositories to make the proper CUDA release available (assuming this was not already there by default). CUDA itself does not care about the carrier board so long as the iGPU is understood, and this in turn is not dependent upon carrier board.
Installing CUDA separately after moving the o/s to SD card is a solution to insufficient space. This also is unrelated to installing CUDA other than having enough space. Was your install ever limited by lack of disk space? I could see a proper install failing as it ran out of space, and if the error was not noted, then it would “appear” to be a lack of CUDA due to incompatibility, but in reality it would be nothing more than needing more disk space. FYI, what do you see from “df -H -T /”?
Incidentally, booting to external media is far more complicated than mounting spare disk space on “/usr/local”, and offloading all of that content to there (and most of the NVIDIA optional content goes there; this is where the most space is required). Do you have an external partition that you are not using, e.g., an eMMC model with SD card slot or NVMe or m.2 storage?