Have written C++-ish wrappers for the CUDA Runtime API - wanna try them out?

[hazelnutvt04]

Could I get an example of how to use Managed Memory and a non-default stream? I am using a Jetson Xavier which has a shared GPU memory and CPU memory. I was initially using zero copy, but switched to using Managed Memory per recommendation here:

Zero-copy still copy data?

Hi, Pinned memory can be shared between CPU and GPU but the performance may not always fast. The zero-copy allocates the physical location of memory is pinned in the CPU system memory. So, a program may have fast or slow access to it depending on where it is being accessed from. It’s recommended to use unified memory instead. CUDA driver can automatically handle the synchronization and pick a better location for you. Here is our document for Jetson memory system for your reference: https:…

Then I have a pipeline of kernels based on this recommendation:

Overall, I want the cuda-api-wrapper equivalent of:

cudaMallocManaged( (void**)input_raw_ptr, num_bytes, cudaMemAttachHost );
cudaMalloc( (void**)intermediate_data1_raw_ptr, num_immed_bytes );
cudaMalloc( (void**)intermediate_data2_raw_ptr, num_immed_bytes );
cudaMallocManaged( (void**)output_raw_ptr, num_output_bytes );

// No cudaMemcpy HtoD needed.

my_kernel1<<< blocks_per_thread, threads_per_block, 0, stream >>>( immed_data1_raw_ptr, input_raw_ptr, num_vals );
my_kernel2<<< blocks_per_thread, threads_per_block, 0, stream >>>( immed_data2_raw_ptr, immed_data1_raw_ptr, num_immed_vals );
my_kernel3<<< blocks_per_thread, threads_per_block, 0, stream >>>( output_raw_ptr, immed_data2_raw_ptr, num_output_vals );

cudaStreamAttachMemAsync(stream, output_raw_ptr, 0, cudaMemAttachHost);
cudaStreamSynchronize(stream);

// No cudaMemcpy DtoH needed.

// Use output_raw_ptr data in host code