I use a GPU for deep learning simulation.
The GPU is NVIDIA Tesla K20m and it has a large heatsink but no cooling fan.
When I simulate deep leaning with python for more than 5 minutes,
the temperature of GPU is over 94 degree Celsius(201 degree Fahrenheit).
I put the GPU in normal desktop PC with 6 system coolers at my office.
Sometimes I found a message like “GPU driver can not be found” during performing ‘watch -n 10 nvidia-smi’.
Is there any solution for changing mode like from ‘full performance mode’ to ‘prevent overheat mode’?
p.s. I try to put more cooling fans around the GPU.
One way to put cooling fans under the GPU on the PCI slot.
The other way, I’ll buy a blow fan and shroud duct for the GPU.
The Tesla K20m is a passively-cooled GPU designed for installation in a server enclosure that provides adequate air flow, typically provided by banks of high-RPM fans and appropriate ducting. Integrators know how to provide sufficient airflow in such an enclosure and they sell servers with the GPU pre-installed. NVIDIA maintains a list of partners that sell such systems: [url]https://www.nvidia.com/en-us/data-center/where-to-buy-tesla/[/url]
The Tesla K20m is not a product targeted at consumers and support for it is designed to be provided by the integrators that sell systems with this GPU. For workstations, there is the actively-cooled K20c that has an integrated fan and a matching shroud.
If you search the internet, you should be able to find people showing off their projects building ducts and funnels from sheet metal or other materials that they attach to the K20m and other passively-cooled GPUs, and then use large fans to direct massive airflow over the heatsinks of those GPUs. In at least some cases, that seems to work. Here is an example using a Tesla K80 (this is not an endorsement on my part, proceed at your own risk):
I assume your current approach using lots of fans is mostly just agitating the air and providing an insufficient and probably very turbulent air flow. What you need is a mostly laminar flow along the fins of the heatsink(s), moving an adequate volume of air per time unit. My memory is hazy, but about 50 cfm should do it.