Simulation slows down after a while

Hi, I am using IsaacLab with IsaacSim 4.1.0. When running the simulation, the frame rate is relatively high at the beginning (around 24fps), but quickly drop to a low level (around 8fps). The GPU usage shows no abnormallity. How to fix that?

Have you observed this performance drop issue while following any specific tutorials or sample scenes in Isaac Sim?

I am building my own humanoid locomotion RL training environment with IsaacLab, based on the provided environment in Environments — Isaac Lab documentation. Only isaaclab and rsl_rl code is changed, not one line of isaacsim.
Any idea on what may lead to this situation?

Did you observe the frame rate slowdown without making any code changes?

No. But I did make a lot of changes on the code, any guess or possible reason for this situation?