GStreamer 1.20.0 seems to break the hardware accelerated encoder, so it does not work flawlessly as I thought. However, I managed to get the rtspsrc element to work with the Dahua camera in GStreamer 1.14.5 by setting short-header to true.
For GStreamer to pick the right plugins GST_PLUGIN_PATH must be set too.
export GST_PLUGIN_PATH="/opt/gstreamer/lib/$(uname -m)-linux-gnu/gstreamer-1.0"${GST_PLUGIN_PATH:+:${GST_PLUGIN_PATH}}