I generally do this via a script, and start at a higher number, then pause, then drop down the lower number.
Also you are right there are limits to how low you can go. On my cards it is
set “25” # Gigabyte RTX 2080 TI Gaming OC - GV-N208TGAMING OC-11GC
set “39” # Gigabyte RTX 3080 TI Gaming OC - GV-N308TGAMING OC-12GD
Anything lower, and it jumps all over the place, and doesn’t set at the number I tell it to go to.