No package install possible after RT-Patches


since I didn’t know how to apply the rt-patches with the SDK manager… I followed the steps here and used Jetpack 4.3 to install the OS image.
Now I get a fully preemtable kernel but I can’t install the NVIDIA components (CUDA) on the target.
I have a Xavier AGX.
When I try to install them with SDK, my target device is detected but when I hit INSTALL the give the sudo password on my host, I’m asked for the ip, login and password on the target.
When I give those I get the following message:

Could not connect to the target device. Verify that:
1. The device is connected to this host machine with a USB cable.
2. Ubuntu 'System configuration wizard' is completed on the device.
3. Jetson's Ubuntu OS is up and running.
4. Jetson USB device mode service is started successfully.
 * See readme file in the auto-mounted "L4T-README" folder for more details.