I wasn’t aware you were headless.
Normal behaviour is any application that access a card primes the card to high performance state initially and after a period of time if there’s no activity, it drops clocks speeds and PCIe Gen and width down, to reduce power.
Your situation seems to crop up quite a bit. Some people have had success reducing monitor refresh rates. An example post is here.