Understanding weirdness with GPUMemoryTransferRateOffset

I am running 375.66 on Ubuntu 16.04 LTS.
The hardware is 6x GTX 1070s.
I am attempting to overclock the memory using nvidia-settings.

The command I am using is: nvidia-settings -a [gpu:{n}]/GPUMemoryTransferRateOffset[3]=1400 where {n} is the iteration over all GPUs.

(I have set GPUPowerMizerMode=1)

As observed in NVIDIA X Server Setting (see linked image below) the vast majority of the time (>95%) the GPUs are in Performance Level 2 during the normal workload (this is a mining rig). If I “locked” into level 2 I could overclock higher. Unfortunately, <5% of the time it changes to Performance Level 3 and the transfer rate offset (e.g., 1400), which appears to apply the same to each Performance Level, then exceeds what the memory can handle and I get errors.

For example, as per the linked image, the offset is 1400 which moves the Performance Level 2 effective transfer rate to 9004Mhz. This is rock solid and stable. However, if whatever logic that is working in the background pops to Level 3, it applies 9408Mhz and that is definitely not stable. Because PL3 happens occasionally I cannot keep the GPUs at +1400. So, I am not able to squeeze all the performance I need.

I figured the way to fix this would be the following:
nvidia-settings -a [gpu:{n}]/GPUMemoryTransferRateOffset[[b]2[/b]]=1400 nvidia-settings -a [gpu:{n}]/GPUMemoryTransferRateOffset[3]=1000

Where Performance Level 2 would get the full +1400, but Performance Level 3 would get less (but end up at the same effective transfer rate).

Unfortunately, GPUMemoryTransferRateOffset[2] has no affect when set in nvidia-settings. It appears only GPUMemoryTransferRateOffset[3] does anything.

So, I want one of two solutions:

  1. For the settings to work as I thought they would, per above
  2. For a way to lock into Performance Level 2 or 3 so I can set the offset and not worry about the level moving.

Can anyone help me figure out how to accomplish this?

Thanks!!!

(image: http://imgur.com/VmUKsi2 )

Don’t now if anyone ever tried this:
https://devtalk.nvidia.com/default/topic/1010856/linux/how-to-force-performance-level-3-driver-381-22-with-gtx-1080-ti-/post/5165034/#5165034