Python cv2.cuda_GpuMat <-> cv2::cuda::GpuMat C++? (pyBind11)

In C++ I receive an image from a camera into a cv2::cuda::GpuMat structure. I would further like to use this GpuMat under Python. cv2.cuda_GpuMat seems to be the matching structure.
I am using pyBind11 to call the C++ methods from python. Is there a way to fill a python structure with the according cuda pointers (without copying the whole data) in order to use it in python (opencv-python with cuda support)?