Learning FPS doesn't increase as much as expected

Hello,

I’ve been working on OmniIsaacSim with a 2070 which was quite slow. Recently I bought a 4080 Super which is supposed to be up to 3.5X faster (rate I observed on diffusion problems).

However on Isaac SIM, I’ve only observed increases up to 1.7 X
Ant 4080 : 423324
Ant 2070 : 325000

FrankaCabinet 4080 : 179061
ISAAC Franka 2070 : 100000

Kraby (my custom env) 4080 : 33000
Kraby (my custom env) 2070 : 21000

4080 DIFFUSION 12.8 it/s - 1:15 full gen
2070 DIFFUSION 3.85 it/s - 4:25 full gen

Would you have any comment on this ? What would you recommend me to unlock full potential of my new GPU ? What tricks do you have to improve performance on Isaac SIM ?

Regards,
Roch

EDIT: quick note, this is 2022.2…0 release