HOWTO make CUDA 1.1 work on FEDORA 8. some sneaky problems fixed.

Hi.

I’ve had some trouble getting cuda 1.1 beta to run on Fedora 8, and I thought I could share my experiences here.

I’ve been able to successfully run bitonic sample, and that seems like a big victory to me. Yes, I’m a bit novice when it comes to linux.

I’ve performed these steps over several days (little precious free time, for this, you know), so I may have forgotten something. The bits I tell are those “holy s**t” moments that stick in memory when you finally understand what the hell is going wrong.

Steps:
a) Get a linux distribution that seems very similar to one of the cuda-supported distributions and that will install on my machine out of the box. Apparently, some linux distributions are fooled by my brand-new hw (RAID 0 bios seems to be to blame). Since cuda 1.1 supports fedora 7 (which won’t install on my machine out of the box) and fedora 8 does install on my machine, fedora 8 is the winner here. And no, I won’t change to this or that distribution or do some complicated installing gym, thank you, not now I’ve been able to run SOME cuda code.

Make sure you install kernel source, It’ll be needed later.

b) Get and install appropiate driver as advised on the downloads page for cuda 1.1 beta. For my machine, I had to change to the directory where kernel source is installed and do “make menuconfig”, or whatever option is required to configure your kernel source with a menu. I just got into the menu and left normally. On my machine, this was required for a kernel module compilation to proceed further.

Of course, in order to install the driver you must NOT be running an X server, something Fedora 8 insists on making difficult. In order to not run it, I had to follow instructions found at

http://www.perpetualpc.net/command_line_login.html

c) Install toolkit and SDK. NVIDIA provides instructions on this.

d) Set PATH, LD_LIBRARY_PATH. Do so by editing .bash_profile on your home folder. Mine looks like this:

.bash_profile

Get the aliases and functions

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

User specific environment and startup programs

PATH=$PATH:$HOME/bin:/usr/local/cuda/bin

export PATH

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib

export LD_LIBRARY_PATH

e) Compile the examples, doing make on the sdk’s root folder.

f) Try to run one of the samples (ie, bitonic) from the command line, and get some error message about libcudart. Do some ldd gym in order to test wether bitonic is finding all shared libraries. Yes, it can find everything. Do the same with libcudart.so. Again, all is there. Be puzzled, understand nothing, run startx to get into the web. Notice a report from SELINUX saying that there’s been some kind of problem. Open it, and realize that SELINUX is preventing libcudart.so from loading because it’s doing something that is potentially dangerous. It even provides details of what to do in order to allow libcudart to load - run this as root:

chcon -t textrel_shlib_t /usr/local/cuda/lib/libcudart.so.1.1

If anyone from NVIDIA is interested in fixing these problems, they will probably request details on what SELinux has to say about their libcudart.so. I wish I knew how to access these details, but my knowledge on selinux is very close to 0. In any the description of the problems is in Spanish (as is my system), so a translation would probably be in order,

I will try to access this info, and provide more details in a later post.

well, good night.

Saludos,

apano

this is what SELINUX has to say about llibcudart.so.1.1. Some of the original text was in Spanish. I’ve translated it back into English - I was not certain on how to translate some bits, which are clearly explained.

Summary
SELinux is denying /home/jorge/NVIDIA_CUDA_SDK/bin/linux/release/bitonic permission to load /usr/local/cuda/lib/libcudart.so.1.1that requires “moving around text” (not certain on how to translate this. Apparently libcudart needs to move some text around and selinux dislikes that).

Detailed description
Application /home/jorge/NVIDIA_CUDA_SDK/bin/linux/release/bitonic tried to load /usr/local/cuda/lib/libcudart.so.1.1 which requires “moving text around”. This is a potential security problem. Most libraries don’t need this permission. And some more long details I don’t consider important.

Allowing access (in English in the original ;-) )
If you trust /usr/local/cuda/lib/libcudart.so.1.1 to run correctly, you can change the file context to textrel_shlib_t. “chcon -t textrel_shlib_t /usr/local/cuda/lib/libcudart.so.1.1” You must also change the default file context files on the system in order to preserve them even on a full relabel. “semanage fcontext -a -t textrel_shlib_t /usr/local/cuda/lib/libcudart.so.1.1” The flollowing command will allow this access:
chcon -t textrel_shlib_t /usr/local/cuda/lib/libcudart.so.1.1

Additional info:
Source context: system_u:system_r:unconfined_t:s0
Destination context: system_u:object_r:lib_t:s0
destination objects: /usr/local/cuda/lib/libcudart.so.1.1 [ file ]
Afected RPM packages:
Policies RPM: selinux-policy-3.0.8-56.fc8
SELinux Active: True
Tipo de Política: targeted
MLS Activado: True
Modo Obediente: Enforcing
Nombre de Plugin: plugins.allow_execmod
Nombre de Equipo: localhost.localdomain
Plataforma: Linux localhost.localdomain 2.6.23.1-49.fc8 #1 SMP Thu Nov 8 21:41:26 EST 2007 i686 i686
Cantidad de Alertas: 5
First Seen: vie 23 nov 2007 21:58:57 CET
Last Seen: vie 23 nov 2007 22:25:47 CET
Local ID: db825643-1bb9-419c-a6a8-aab09e4a04de
Números de Línea:

Raw auditing data
avc: denied { execmod } for comm=bitonic dev=dm-3 egid=500 euid=500 exe=/home/jorge/NVIDIA_CUDA_SDK/bin/linux/release/bitonic exit=-13 fsgid=500 fsuid=500 gid=500 items=0 path=/usr/local/cuda/lib/libcudart.so.1.1 pid=4401 scontext=system_u:system_r:unconfined_t:s0 sgid=500 subj=system_u:system_r:unconfined_t:s0 suid=500 tclass=file tcontext=system_u:object_r:lib_t:s0 tty=tty1 uid=500

By the way, the fft library also has this problem.

Good night.

Saludos,

apano

In my past experience (with both x86_64 and i386 International and Italian localized Fedoras - 6,7 and 8) the best thing you can do to let CUDA work without any trouble at all is to disable SELinux. Moreover, if your main aim is to develop and debug your apps, SELinux is a main pain for every relocatable code (like most of closed source drivers and apps, for example).

My tip is to don’t disable SELinux on Fedora’s installation (it takes long time to remove metadatas in your whole FS) but to act on /etc/sysconfig/selinux this way:

(from Fedora’s wiki)

SELINUX=<enforcing|permissive|disabled> —
Defines the top-level state of SELinux on a system.

* enforcing — The SELinux security policy is enforced.
* [b]permissive [/b]— The SELinux system prints warnings but does not enforce policy. This is useful for debugging and troubleshooting purposes. In permissive mode, more denials will be logged, as subjects will be able to continue with actions otherwise denied in enforcing mode. For example, traversing a directory tree will produce multiple avc: denied messages for every directory level read, where a kernel in enforcing mode would have stopped the initial traversal and kept further denial messages from occurring.
* disabled — SELinux is fully disabled. SELinux hooks are disengaged from the kernel and the pseudo-file system is unregistered. 

Put your choice on permissive (be careful with log files - they become larger and larger for every CUDA program run) or disabled to completey shut down SELinux. Reboot and enjoy your de-selinuxed environment…

PS: for guys… DON’T TRY THIS AT WORK! :P Production servers should keep SELinux on!

In case anyone was wondering… I have Fedora 8 x86_64 installed on my HP dv6000 notebook with the 8400M GPU. I am able to compile all the SDK examples w/o error or any special configurations. Everything you need to get CUDA (Fedora 7 x86_64 Toolkit) working on Fedora 8 in contained in this thread. :wave:

You can just keep RAID0 disabled in BIOS, since it is usually just a software-RAID. If you want RAID-0 it’s better to use the linux software RAID

You can also just enter in a terminal (as root) telinit 3, install the driver, telinit 5

There is even a ‘cleaner’ system-wide way. in /etc/ld.config.d/ you will find single files that contain linker directories.

So make a cuda file and add LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib

And you can do the same for the PATH in /etc/profile.d/

And about SELinux, I have also turned it off on my system (since I am not connected to the internet) I will try your solution on my home system soon.

Please note that I had problems at first with placing the path and ld path in .bash_profile. All my problems were fixed when I transfered the lines to .bashrc… just a thought