Hi,
There is limitation in hardware engines. So for running with OpenCV, it consumes much CPU loading. Please check discussion in
[Gstreamer] nvvidconv, BGR as INPUT - #2 by DaneLLL
Since you need close-to-4K resolution, the memory copy may dominate the performance. Please execute sudo nvpmodel-m 0 and sudo jetson_clocks. These commands enable CPU cores at max clocks always and should bring performance improvement.
Or you may consider to run jetson_multimedia_api and apply OpenCV functions to NvBuffer.