Isaac Sim + omniisaacgymenvs gets a lower reward when headless=False compared to headless=True

After training the FrankaCabinet example the agent performs worse when rendering is applied.
When test=True and headless=True, thus no rendering, the agents gets around 2660 as reward and is able to open the drawer. However when test=True and headless=False it gets around 2300 points and is not able to open the drawer fully.
How can this issue be fixed?