Hello kesong and motyaedu and sorry for my inactivity.
Lately I have noticed that the problem has disappeared from most of the cameras I use even though I did not change the pipeline. The fact that more cameras are working correctly without changes in the pipeline would suggest that the connection to the cameras might have been bad earlier. So, forcing TCP should also solve this issue as motyaedu suggests and I will try that out once I have the time. But since my pipeline is now mostly working and I also have a workaround this issue is now solved for me. I will post an update if I find out something new when I try out the TCP.
Thanks to both of you for the help.