CUDA runs on Quadro card, hangs on Tesla card [CentOS 5.5 + Older Hardware]

I am running a Quadro card (nvs 300) and a Tesla card (c1060) in a Dell T3400 workstation with a duo quad core PC on Centos 5.5 (I also tried RHEL 6.3, but switched to see if it would solve the issue):

CUDA installation seems to work perfectly on the Quadro card, but the Tesla card hangs on pretty much anything. deviceQuery runs just fine, but other than that, nothing else works. When it hangs, it is sometimes impossible to use the command prompt, but otherwise I can control-c easily. It usually hangs in pthread_mutex_lock() when using cuda-gdb.

Most recent things I’ve tried is editing boot parameters based on the luck others have had using a 2.6 SMP kernel. These are the parameters I changed: pci=nommconf idle=poll maxcpus=1 clocksource=hpet.

Here is the call trace from demesg:

BUG: soft lockup - CPU#0 stuck for 10s! [clock:3019]
CPU 0:
Modules linked in: autofs4 hidp rfcomm l2cap bluetooth lockd sunrpc ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables ipv6 xfrm_nalgo crypto_api loop dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac lp nvidia(PU) snd_hda_intel snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss serio_raw snd_pcm sr_mod floppy i2c_i801 cdrom i2c_core tg3 sg shpchp snd_timer snd_page_alloc parport_pc snd_hwdep parport snd soundcore pcspkr dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 3019, comm: clock Tainted: P 2.6.18-194.el5 #1
RIP: 0010:[] [] :nvidia:_nv012319rm+0x36/0x3a
RSP: 0000:ffffffff80448d10 EFLAGS: 00000206
RAX: 0000000000000c88 RBX: ffff81011aaaac80 RCX: 0000000000003220
RDX: ffffc20003080000 RSI: ffff810119d20008 RDI: ffff8101292be008
RBP: ffffffff80448c90 R08: ffff81011aaaac8c R09: ffff8100010503d4
R10: ffff81011aaaada8 R11: ffffffff80172534 R12: ffffffff8005ec8e
R13: 00000000003040b0 R14: ffffffff8007922b R15: ffffffff80448c90
FS: 00002ad996cf7860(0000) GS:ffffffff803cb000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002ad996ef2e30 CR3: 000000011a17c000 CR4: 00000000000006e0

Call Trace:
[] :nvidia:_nv003411rm+0x481d/0xb184
[] :nvidia:_nv006797rm+0x1e5/0x217
[] :nvidia:_nv013682rm+0x180/0xd0f
[] :nvidia:_nv013683rm+0x8ca/0xcfe
[] :nvidia:_nv007246rm+0x29f/0x944
[] :nvidia:_nv007246rm+0x26c/0x944
[] :nvidia:_nv007244rm+0x65/0x6f
[] :nvidia:_nv007254rm+0x1d1/0xb1b
[] :nvidia:_nv011125rm+0x52a/0x868
[] :nvidia:_nv011108rm+0x65/0x8f
[] :nvidia:_nv000792rm+0x120/0x18a
[] :nvidia:rm_isr_bh+0x23/0x73
[] tasklet_action+0x89/0xfd
[] __do_softirq+0x89/0x133
[] call_softirq+0x1c/0x28
[] do_softirq+0x2c/0x85
[] do_IRQ+0xec/0xf5
[] ret_from_intr+0x0/0xa

NVRM: os_schedule: Attempted to yield the CPU while in atomic or interrupt context
NVRM: os_schedule: Attempted to yield the CPU while in atomic or interrupt context
NVRM: os_schedule: Attempted to yield the CPU while in atomic or interrupt context
NVRM: os_schedule: Attempted to yield the CPU while in atomic or interrupt context

If you have any idea of what this is about and you can help me out, you’d basically make me the happiest person alive.

Seems to me more like a hardware problem, like not enough power or the MB.

I agree, could be a power or motherboard issue. Try the Tesla card on another machine if possible to make sure that is not the problem.

Not Tesla related, but a few days ago we received a GaGe digitizer card and it refused to work correctly on a Dell Optiplex 9020 (after I painstakingly drilled the rivets to get the hard drive cage out so the card would fit). However, it worked fine on a previous model (Dell Optiplex 980). Another suggestion would be to update the BIOS on the machine you have and see if that makes any difference. Seems like the latest BIOS is A14 for that system: