Gradualy increased memory usage when use gstreamer + opencv

I’ve seen similar issue on Xavier R32.4 with h264parse before nvv4l2decoder, as reported here. You may thus try to remove h264parse.