Gstreamer RSTP server with nvcompositor latency increases as time goes by

Please share which release you are using. And for further check, you may try videotestsrc, use USB camera to try v4l2src, or use Bayer camera to try nvarguscamerasrc. See if the issue is seen with other types of sources.

And you may also try to run VIC engine at maximum clock:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL
This enables nvcompositor in maximum throughput.