How to print/inspect objects in GPU memory

Looking for information on how to display contents of the GPU memory via nvidia-gdb or any other debugging tool. I have studied the cuda/nvidia-gdb documentation, and searched for applicable posts here and on Stack-Overflow, have not found anything yet.
In particular, I am looking to inspect GpuMat objects stored in GPU memory as I process a set of OpenCV and proprietary kernels.
Any pointer to existing documentation would be very helpful.
My system is a Dell Laptop + A5000 GPU, with Ubuntu 2004 and Cuda 11.6.


There are certainly postings on both stack overflow as well as these forums that mention using cuda-gdb for inspecting memory.

I’m not aware of any that are specific to GpuMat objects. I’m not an OpenCV expert, and OpenCV is open source, so you should be able to deduce a method to symbolically inspect data in a GpuMat object. For a non-symbolic method, you could print out the value of the GpuMat data pointer and use that. Of course cuda-gdb isn’t going to know anything about data organization or padding within the buffer/object.

Unit 12 of this online training series discusses CUDA debugging with a section on usage of cuda-gdb. And of course there is the documentation.

ThX for the response.

  1. I had not found the resources you mention above. ThX! Will read.

  2. I should have added the following: I did “normal nvidia-gdb debugging”, when at a breakpoint I first printed the Mat and GpuMat objects, and I noticed that when I uploaded and downloaded a Mat to/from the GPU, the Mat objects in host memory before upload and after download had non-zero data in it, while the corresponding GpuMat object in the GPU only displayed zeroes. I was using commands like “p /x <my_Mat>.datastart[0]@50” and “p /x <my_GpuMat>.datastart[0]@50”, or also “x /50x <addr_of_datastart>”.


to inspect device memory, you must be stopped at a breakpoint in GPU device code.