Hi ManuKlause,
When create the cv::Mat first, then the GpuMat from it, things work. This tells that you have host memory from your camera input, so it must be explicitly uploaded to GPU memory before use. You can do some hiding of this latency if you use streams with asynchronous copy. Here is a nice presentation form GTC 2013 they should review for more info:
[url]http://on-demand.gputechconf.com/gtc/2013/webinar/gtc-express-itseez-opencv-webinar.pdf[/url]
Hope this could help on your case.
Thanks