I think I am right in saying that the various toolkit versions differ only based on what versions (and naming conventions) of underlying libraries like glibc and tools like the compiler and linker are used to build them. The underlying kernel version shouldn’t have any bearing within reason (the kernel version specific material is supplied in the driver package). So if you have Ubuntu 9.04 with the standard Ubuntu 9.04 toolchain, then you should use the 9.04 release of the toolkit.
If you’re using ubuntu 9.04 but with 8.10’s kernel, then you’re in a gray area. The kernel that you’re using definitely can have an impact on overall stability. CUDA functionality isn’t tied to a kernel version though.