How to get memory info when card working?
real-time monitoring ?
Similar to the task manager on windows???
cuMemGetInfo can’t working in real-time.
how to ?
Thanks…
I use Rivatuner with a GTX285 and it works fine.
I’m still hoping that nVidia will add some Windows performance counters to their driver one of these days so things like processor and memory usage can be viewed in real-time and also logged if you want.
What are some practical use-cases for this?
Well, it would let you make some GPU usage meters, just like the ones for CPU that you see all the time…
Well, for one (like Keldor said) you could add a GPU performance monitor to the Windows Resource Monitor. If you have access to a Vista box, open the Task Manager, go to the performance tab, then hit the ‘Resource Monitor’ button. There’s already sections in there for CPU/Disk/Network/Memory and they show which programs are using those, how much utilization for each, etc. I think it would be good to have a ‘GPU’ section available in there to see which programs are making use of the GPU and how much. As more and more programs start to leverage GPU programming, it will be important to be able to monitor how programs are using it.
Also, by adding the performance counters into the driver (and making them available through the Resource Monitor in Vista and the Performance monitor under Windows Server), anyone that is using a bunch of headless servers with Teslas (or whatever) will be able to remotely monitor and log the performance of the GPU’s in their servers, to make sure that everything is running as smoothly as possible. If a computer hangs because of a GPU bug, you would be able to tell right away from the performance counters (you would see that no memory is being transferred, the GPU is pegged at 100% due to a bad loop, etc.)
EDIT: Here’s a link to the performance counter API: http://msdn.microsoft.com/en-us/library/aa373083(VS.85).aspx. Version 1.0 is for XP, Server 2003, etc., and version 2.0 (which is supposedly easier to work with as well) is for Vista, Server 2008, Windows 7, etc.
Also, now that I’m thinking about it…it would also be very helpful (mostly for the HPC-type people) to have the driver (or perhaps just the runtime) record things like kernel launches, driver errors (e.g. CUDA_ERROR_OUT_OF_MEMORY), and so on into the Windows Event Log so that system administrators have a better way of finding out what went wrong (when something inevitably does). Any companies writing CUDA-enabled programs would also have a better way to record and analyze data if their programs could read this kind of information out of the event log in case of a system crash (since there won’t be any other way to save the data that caused the crash before the kernel panic/BSOD kicks in).
EDIT: Link to the event log API: [url=“Windows Events - Win32 apps | Microsoft Docs”]Microsoft Docs - Developer tools, technical documentation and coding examples
On the Linux side of things, it would be nice from an administration perspective to have a “gputop” that could show me which host processes have CUDA contexts open on which GPUs, what percentage of the time each GPU is actively being used, and how much device memory each context has allocated.