I have a strange thing that is happening on one of 3 cards in a system that I am using for Etherume Mining.
For some reason the Quadro K5200 GPU clock drops from 875MHz to 770MHz and sometimes 666MHz
Using nvidia-smi to get NVSMI LOG I see that under Throttle Reasons Applications Clocks Setting is Active.
My test setup:
Dell precision T7500 Intel Xeon X5687 24GB Ram
Windows 7 Etnterprise Service Pack 1
GPU’s
0 Quadro K5200
1 Quadro K5000
2 Quadro 5000
Nvida Driver Version 364.72
All three have dedicated miners running on the GPUs called ethminer-0.9.41-genoil-1.0.7 which is designed to run CUDA 7.5
The two other cards do not have this problem at all so I am not sure that it is actually the application that is causing this. But I attempted to restrict changing of the –ac
nvidia-smi -acp 1 -i 0
Applications clocks commands have been set to RESTRICTED for GPU 0000:03:00.0
All done.
This did to change help.
The card is not getting warmer than 60deg.
The power limit is also not the issue. To test I changed the max power to 165W. I never see it over 130W
I noticed that the voltage of the GPU drop from 1037mV @ 875Mhz 125W to 962mV@770MHz@ 107.10W
I attempted to reset the rac using this command:
nvidia-smi -rac
Setting applications clocks is not supported for GPU 0000:03:00.0.
Treating as warning and moving on.
Using EVGA PrecisionX 16 I can reset it between jobs but it just re-activates after a few seconds.
What is causing the Throttle Applications Clocks Setting to activated?
here are a view logs
==============NVSMI LOG============== Fast 875MHz
Timestamp : Tue May 03 12:28:17 2016
Driver Version : 364.72
Attached GPUs : 3
GPU 0000:03:00.0
Product Name : Quadro K5200
Product Brand : Quadro
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 1920
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : 0323914019331
GPU UUID : GPU-05dc7d4a-3f72-ca0b-dc47-683f4c8ddbdd
Minor Number : N/A
VBIOS Version : 80.80.53.00.08
MultiGPU Board : No
Board ID : 0x300
GPU Part Number : N/A
Inforom Version
Image Version : 2081.0502.00.01
OEM Object : 1.1
ECC Object : 3.0
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
PCI
Bus : 0x03
Device : 0x00
Domain : 0x0000
Device Id : 0x103C10DE
Bus Id : 0000:03:00.0
Sub System Id : 0x109510DE
GPU Link Info
PCIe Generation
Max : 2
Current : 2
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : N/A
Rx Throughput : N/A
Fan Speed : 70 %
Performance State : P0
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
Sync Boost : Not Active
Unknown : Not Active
FB Memory Usage
Total : 8192 MiB
Used : 1541 MiB
Free : 6651 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 15 MiB
Free : 241 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : N/A
Decoder : N/A
Ecc Mode
Current : Disabled
Pending : Disabled
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending : N/A
Temperature
GPU Current Temp : 60 C
GPU Shutdown Temp : 100 C
GPU Slowdown Temp : 95 C
Power Readings
Power Management : Supported
Power Draw : 126.51 W
Power Limit : 165.00 W
Default Power Limit : 150.00 W
Enforced Power Limit : 165.00 W
Min Power Limit : 10.00 W
Max Power Limit : 165.00 W
Clocks
Graphics : 875 MHz
SM : 875 MHz
Memory : 3004 MHz
Video : 540 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 875 MHz
SM : 875 MHz
Memory : 3004 MHz
Video : 540 MHz
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 4
Type : Insufficient Permissions
Name : Insufficient Permissions
Used GPU Memory : Not available in WDDM driver model
Process ID : 9724
Type : C
Name : c:\ether\CUDA 7.5\ethminer.exe
Used GPU Memory : Not available in WDDM driver model
==============NVSMI LOG============== Slow 770MHz
Timestamp : Tue May 03 12:22:59 2016
Driver Version : 364.72
Attached GPUs : 3
GPU 0000:03:00.0
Product Name : Quadro K5200
Product Brand : Quadro
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 1920
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : 0323914019331
GPU UUID : GPU-05dc7d4a-3f72-ca0b-dc47-683f4c8ddbdd
Minor Number : N/A
VBIOS Version : 80.80.53.00.08
MultiGPU Board : No
Board ID : 0x300
GPU Part Number : N/A
Inforom Version
Image Version : 2081.0502.00.01
OEM Object : 1.1
ECC Object : 3.0
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
PCI
Bus : 0x03
Device : 0x00
Domain : 0x0000
Device Id : 0x103C10DE
Bus Id : 0000:03:00.0
Sub System Id : 0x109510DE
GPU Link Info
PCIe Generation
Max : 2
Current : 2
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : N/A
Rx Throughput : N/A
Fan Speed : 72 %
Performance State : P0
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Active
SW Power Cap : Not Active
HW Slowdown : Not Active
Sync Boost : Not Active
Unknown : Not Active
FB Memory Usage
Total : 8192 MiB
Used : 1541 MiB
Free : 6651 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 15 MiB
Free : 241 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : N/A
Decoder : N/A
Ecc Mode
Current : Disabled
Pending : Disabled
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending : N/A
Temperature
GPU Current Temp : 60 C
GPU Shutdown Temp : 100 C
GPU Slowdown Temp : 95 C
Power Readings
Power Management : Supported
Power Draw : 107.10 W
Power Limit : 165.00 W
Default Power Limit : 150.00 W
Enforced Power Limit : 165.00 W
Min Power Limit : 10.00 W
Max Power Limit : 165.00 W
Clocks
Graphics : 771 MHz
SM : 771 MHz
Memory : 3004 MHz
Video : 540 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 875 MHz
SM : 875 MHz
Memory : 3004 MHz
Video : 540 MHz
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 4
Type : Insufficient Permissions
Name : Insufficient Permissions
Used GPU Memory : Not available in WDDM driver model
Process ID : 9724
Type : C
Name : c:\ether\CUDA 7.5\ethminer.exe
Used GPU Memory : Not available in WDDM driver model