I am exploring how the usage of CUDA impacts my Linux system. I wanted to test out how memory was used when pinned and pageable memory was used on the host. To do this, I wrote the following little program and executed the Linux “free” command and “ps” with rss and vsize options each time the program paused.
Here’s the code :
#define MEM_SIZE_BYTES 32000000
void runTest(int argc, char** argv)
{
CUT_DEVICE_INIT();
void *hostArrayA = NULL;
printf("Pause...\n");
sleep(30);
printf("Allocating pinned memory for %d bytes\n",MEM_SIZE_BYTES);
// Allocate page-locked memory.
CUDA_SAFE_CALL(cudaMallocHost((void **) &hostArrayA,MEM_SIZE_BYTES));
printf("Pause...\n");
sleep(30);
printf("Freeing up pinned memory\n");
CUDA_SAFE_CALL(cudaFreeHost(hostArrayA));
printf("Pause...\n");
sleep(30);
printf("Allocating pageable memory for %d bytes\n",MEM_SIZE_BYTES);
hostArrayA = NULL;
hostArrayA = malloc(MEM_SIZE_BYTES);
if (hostArrayA==NULL)
{
printf("Error mallocing\n");
}
printf("Pause...\n");
sleep(30);
printf("Freeing up pageable memory\n");
free(hostArrayA);
printf("Pause...\n");
sleep(30);
printf("Exiting program\n");
}
Here’s the output :
[font=“Courier”]
PID PPID RSS VSZ %CPU %MEM CMD
7489 6846 9516 20932 0.1 0.4 test_mem
total used free shared buffers cached
Mem: 2072916 909408 1163508 0 34084 483316
-/+ buffers/cache: 392008 1680908
Swap: 2096440 0 2096440
Allocating pinned memory for 32000000 bytes
Pause…
PID PPID RSS VSZ %CPU %MEM CMD
7489 6846 10704 22012 0.6 0.5 test_mem
total used free shared buffers cached
Mem: 2072916 948464 1124452 0 34088 483312
-/+ buffers/cache: 431064 1641852
Swap: 2096440 0 2096440
Freeing up pinned memory
Pause…
PID PPID RSS VSZ %CPU %MEM CMD
7489 6846 10704 22012 0.3 0.5 test_mem
total used free shared buffers cached
Mem: 2072916 917160 1155756 0 34104 483296
-/+ buffers/cache: 399760 1673156
Swap: 2096440 0 2096440
Allocating pageable memory for 32000000 bytes
Pause…
PID PPID RSS VSZ %CPU %MEM CMD
7489 6846 10708 53264 0.2 0.5 test_mem
total used free shared buffers cached
Mem: 2072916 917168 1155748 0 34124 483276
-/+ buffers/cache: 399768 1673148
Swap: 2096440 0 2096440
Freeing up pageable memory
Pause…
PID PPID RSS VSZ %CPU %MEM CMD
7489 6846 10704 22012 0.1 0.5 test_mem
total used free shared buffers cached
Mem: 2072916 917136 1155780 0 34140 483260
-/+ buffers/cache: 399736 1673180
Swap: 2096440 0 2096440[/font]
As you can see, the used swap space output by the “free” utility never goes above zero. I would have expected this to change when pinned memory was being used. The used Mem increases though. The VSZ output by “ps” in red increases when pageable memory is used, which is what we expect to see. However, it does not increase when pinned memory is used.
My questions :
[1] How does the Linux OS detect that its swap space has gone down when CUDA uses memory in it? Is there a way for me to see this happening?
[2] Is the pinned memory made available to the OS once the CUDA application exits even if it is an abnormal exit (i.e app crashes)?
Thank you for any help/clarification you can provide.
skb