Reading the docs from opencv would probably help you to get the most of it.
In a few words, it depends first on which openCV you intend to use.
For OpenCV2, such as opencv4tegra you will find classes within namespace cv::gpu, such as cv::gpu::GpuMat for a matrix. You have to handle memory allocations in gpu memory and manage host (cpu) to/from device (gpu) transfers. You will also have to link against libopencv_gpu.so
If you are using Opencv3 it will be within namespace cv::cuda, and you will link against libopencv_cuda* libraries such as libopencv_cudaobjdetect.so.
If you are building opencv from source, be sure to activate the cuda/gpu part and set the right architecture options to get best performances.