Pinned memory limit

Accelerated Computing CUDA CUDA Programming and Performance
17

I still hope to find out what causes this problem. As some people suggest, I break the cudaMallocHost() call into two calls: posix_memalign() and cudaHostRegister().

Here’s my new testing program:

#include <iostream>
#include <assert.h>

#include <cuda.h>
#include <cuda_runtime.h>

using namespace std;

int main() {
  size_t count = 0;
  size_t size = 64 * 1024 * 1024 * sizeof(float);
  while (true) {
    void *ptr;
    while (posix_memalign(&ptr, 4096, size) != 0) {
      cout << "posix_memalign failed at " << count * 256 << " MB" << endl;
    }
    cout << "ptr = " << ptr << endl;
    while (cudaHostRegister(ptr, size, 0) != cudaSuccess) {
      cout << "cudaHostRegister failed at " << count * 256 << " MB" << endl;
    }
    count++;
    cout << "Allocated " << count * 256 << " MB" << endl;
  }
}

The program always fails at the cudaHostRegister() call when the allocated memory approaches 4 GB. I’m using Ubuntu 14.04 with CUDA 7.5.

Do you have any ideas how the cudaHostRegister() is implemented? I really hope I can write a simple program to emulate what cudaHostRegister(), but without using CUDA APIs, so that I can find out whether it’s a Ubuntu issue or CUDA issue.

Thank you!
Cui