The solution was to set
aeantibanding=0
in my nvarguscamerasrc element in my Gstreamer pipeline. Maybe this was because of some misconfiguration because we have a custom sensor and the ISP config file is not made for that sensor.
Thank you anyway for your help!