I recently found some rather weird thing on Ubuntu Feisty which could be a problem in later versions of CUDA. To make it clear, AFAIK CUDA runs fine out of the box, but who knows what will happen as CUDA develops further…
The linux-restricted-modules-$KERN_VERSION packages contain an older version of the Nvidia kernel module, which will be build on boot time and placed in /lib/modules/$KERN_VERSION/volatile. The Nvidia installer will place the kernel module (using the same name) in /lib/modules/$KERN_VERSION/kernel/drivers/video. As a result, modprobe will always load the old driver instead of the newer drivers which is not what one might expect after installing the CUDA drivers.
To circumvent this, I did the following:
(1) Disable build of Nvidia module in /etc/default/linux-restricted-modules-common
(2) Remove already built modules:
# rm -rf /lib/linux-restricted-modules/$KERN_VERSION/nv*
Substitute $KERN_VERSION with your kernel version string (at my box currently: 2.6.20-16-generic).