libcudart.dylib install location

Accelerated Computing CUDA CUDA Programming and Performance
1

I would like to ship libcudart.dylib with my software, but I’m having trouble deciding where to install it.

The library has the following path baked into it:

@rpath/libcudart.dylib

Which I believe means the linker will search for it in the normal library path. But it’s a really bad idea for me to install cudart in the system path – what if I ship the 2.3 runtime, and some other app comes along and installs the 2.2 runtime, or vice versa?

I could put it in my application directory, and add that path to DYLD_LIBRARY_PATH. But that’s a bit fragile, especially since my software is a plug-in – the host application might do things to the library path that I have no control over. And if someone installs cudart in /usr/lib, it will be found before my copy, leading to the same problems as above.

With other 3rd-party libs, I’ve typically done the following:

    [*]Make a copy of the library in my application directory.

    [*]Use install_name_tool to change the baked-in path, so it also points to my application directory.

    [*]Link against the copy instead of the original.

This ensures that my software will always use its private copy of the library.

Unfortunately, if I change the install name in libcudart.dylib, it stops working! It can no longer find any CUDA devices, and cudaDriverGetVersion() says the driver version is 0 (cudaRuntimeGetVersion returns the correct value, 2030).

Has anyone successfully shipped a CUDA app to end users on the Mac? How do you go about deploying the CUDA libraries?

2

Shipping libcudart.dylib (and libcufft.dylib and libcublas.dylib, if you use them) from the CUDA Toolkit with your application is indeed the correct approach. You should install them into a location relative to your application’s installation directory rather than to any absolute path in the system for exactly the reasons you mentioned.

We suggest linking with these libraries in such a way that the application binary contains references within the application bundle similar to the following:

[font=“Lucida Console”] @executable_path/…/Library/libcudart.dylib

@executable_path/…/Library/libcufft.dylib

…[/font]

You are correct that install_name_tool is the way to set this up, using install_name_tool to change the LC_ID_DYLIB for the libraries to be redistributed. The full process would look like this:

To change this for libcudart.dylib (after copying it to a new location):

[font=“Lucida Console”]> install_name_tool -id “@executable_path/…/Library/libcudart.dylib” libcudart.dylib [/font]

To verify that this worked, use the otool -D command:

[font=“Lucida Console”]> otool -D libcudart.dylib

libcudart.dylib:

@executable_path/…/Library/libcudart.dylib[/font]

When an application is linked with this modified libcudart.dylib (using the appropriate -L and -lcudart options), the resulting binary will contain the reference “@executable_path/…/Library/libcudart.dylib” instead of “@rpath/libcudart.dylib”. The application bundle must of course be constructed to contain the necessary paths to find libcudart.dylib at location you have chosen.

Note: As of CUDA 3.0, libtlshook.dylib must be redistributed alongside libcudart.dylib. Our suggestion is to place it in the same directory as the redistributed libcudart.dylib. libcudart.dylib contains a reference to “@rpath/libtlshook.dylib” and also has an LC_RPATH entry for “@loader_path/”. In this way, it expects to be able to find libtlshook.dylib in the same directory as libcudart.dylib is itself found. The same treatment as above can be applied to libcufft.dylib.

By way of example, let’s assume we have a very simple hello.cu file, and that we’ll compile it to “hello”, which will depend on a copy of libcudart.dylib in the same directory as the executable. For this example, we’ll skip creating a full application bundle, but we’ll at least want to verify that the libcudart.dylib we’re using is the one in the same directory as the hello executable:

[font=“Lucida Console”]> cd

cp /usr/local/cuda/lib/libtlshook.dylib .

cp /usr/local/cuda/lib/libcudart.dylib .

install_name_tool -id “@executable_path/” libcudart.dylib

nvcc -noprof -I/usr/local/cuda/include -L. -lcudart -o hello hello.cu[/font]

Now if we examine the resulting binary, we can see that it contains a dylib load command that refers to the path containing the executable:

[font=“Lucida Console”]> otool -l hello

[…]

      cmd LC_LOAD_DYLIB

  cmdsize 60

     name @executable_path/libcudart.dylib (offset 24)

[…][/font]

These instructions came directly from our software team, so if you’ve tried these steps (it sounds like you might have) and they’re still not working for some reason, please let me know. I’d also be interested to know whether you’ve tried it with both CUDA 2.3 and CUDA 3.0.

Thanks,

–Cliff