swap space on linux How can I monitor CUDA swap use?

I am exploring how the usage of CUDA impacts my Linux system. I wanted to test out how memory was used when pinned and pageable memory was used on the host. To do this, I wrote the following little program and executed the Linux “free” command and “ps” with rss and vsize options each time the program paused.

Here’s the code :

#define MEM_SIZE_BYTES 32000000

void runTest(int argc, char** argv) 

{

    CUT_DEVICE_INIT();

   void *hostArrayA = NULL;

   printf("Pause...\n");

    sleep(30);

    printf("Allocating pinned memory for %d bytes\n",MEM_SIZE_BYTES);

   // Allocate page-locked memory.

    CUDA_SAFE_CALL(cudaMallocHost((void **) &hostArrayA,MEM_SIZE_BYTES));

   printf("Pause...\n");

    sleep(30);

    printf("Freeing up pinned memory\n");

   CUDA_SAFE_CALL(cudaFreeHost(hostArrayA));

   printf("Pause...\n");

    sleep(30);

    printf("Allocating pageable memory for %d bytes\n",MEM_SIZE_BYTES);

   hostArrayA = NULL;

    hostArrayA = malloc(MEM_SIZE_BYTES);

    if (hostArrayA==NULL)

      {

	printf("Error mallocing\n");

      }

   printf("Pause...\n");

    sleep(30);

    printf("Freeing up pageable memory\n");

    

    free(hostArrayA);

   printf("Pause...\n");

    sleep(30);

    printf("Exiting program\n");

}

Here’s the output :

[font=“Courier”]

PID PPID RSS VSZ %CPU %MEM CMD

7489 6846 9516 20932 0.1 0.4 test_mem

         total       used       free     shared    buffers     cached

Mem: 2072916 909408 1163508 0 34084 483316

-/+ buffers/cache: 392008 1680908

Swap: 2096440 0 2096440

Allocating pinned memory for 32000000 bytes

Pause…

PID PPID RSS VSZ %CPU %MEM CMD

7489 6846 10704 22012 0.6 0.5 test_mem

        total       used       free     shared    buffers     cached

Mem: 2072916 948464 1124452 0 34088 483312

-/+ buffers/cache: 431064 1641852

Swap: 2096440 0 2096440

Freeing up pinned memory

Pause…

PID PPID RSS VSZ %CPU %MEM CMD

7489 6846 10704 22012 0.3 0.5 test_mem

        total       used       free     shared    buffers     cached

Mem: 2072916 917160 1155756 0 34104 483296

-/+ buffers/cache: 399760 1673156

Swap: 2096440 0 2096440

Allocating pageable memory for 32000000 bytes

Pause…

PID PPID RSS VSZ %CPU %MEM CMD

7489 6846 10708 53264 0.2 0.5 test_mem

       total       used       free     shared    buffers     cached

Mem: 2072916 917168 1155748 0 34124 483276

-/+ buffers/cache: 399768 1673148

Swap: 2096440 0 2096440

Freeing up pageable memory

Pause…

PID PPID RSS VSZ %CPU %MEM CMD

7489 6846 10704 22012 0.1 0.5 test_mem

       total       used       free     shared    buffers     cached

Mem: 2072916 917136 1155780 0 34140 483260

-/+ buffers/cache: 399736 1673180

Swap: 2096440 0 2096440[/font]

As you can see, the used swap space output by the “free” utility never goes above zero. I would have expected this to change when pinned memory was being used. The used Mem increases though. The VSZ output by “ps” in red increases when pageable memory is used, which is what we expect to see. However, it does not increase when pinned memory is used.

My questions :

[1] How does the Linux OS detect that its swap space has gone down when CUDA uses memory in it? Is there a way for me to see this happening?

[2] Is the pinned memory made available to the OS once the CUDA application exits even if it is an abnormal exit (i.e app crashes)?

Thank you for any help/clarification you can provide.

skb

Pinned memory is memory-mapped directly from physical memory (through a device node), so it will not show up in those memory statistics for that reason.

And yes, it seems to be freed on crashes and such.

Thank you for your reply.

Is there any way that I can observe the swap space being used by CUDA on a Linux system? I would think that the OS has to know about the use at some level since it should stay away from using it for other purposes. Are there any Linux utilities which will show this information?

Is there any way that I can observe the swap space being used by CUDA on a Linux system (other than the total “used memory” change which would have other things in it also)? I would think that the OS has to know about the use at some level since it should stay away from using it for other purposes. Are there any Linux utilities which will show this information?

Thanks,
skb

Yikes. Sorry about the repeat posts. The database was having hiccups when I posted originally :(

Not that I know of. I think the part of the OS that swaps pages out to swap-disk does not know to which program they belong to. It is usually the least-used memory, so if you are actively using your memory in your program it will not be swapped out unless you are running low on memory.

some things that stood out in a google search:
http://www.cyberciti.biz/tips/linux-resour…ottlenecks.html
http://www.justlinux.com/forum/archive/ind…p/t-148178.html