Subtracting a vector from every column of a matrix

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi everyone,

What would be a fast way to subtract a vector V from every column of a matrix M using (preferably) CUBLAS?

I first tried constructing a matrix A with the same size as M and with each column equal to V, using cublasSetMatrix(). However, wouldn’t it be much faster if I could load V from host to device first, and then construct A? I can’t find a way to do that using CUBLAS. Any suggestions?

Many thanks,
Arno

2

You would be better off just writing a small compute kernel to do the operation directly in device memory. How large are the matrices and vectors?

3

The vector contains 2048 elements (floats) and the matrix is generally 2048 x 65536.

4

At that size, a dedicated kernel definitely makes sense. There is even some hope of being faster than doing it on the host.

5

Many thanks for your advice.

I’m obviously new to CUDA, so I’m not sure which solution would be fastest:

  1. copy the vector V to device (global) memory; build a matrix A with each column equal to V using a kernel; subtract A from M,

or

  1. copy one element of V to shared memory; make a thread block subtract this element from the corresponding row of matrix M; do this for all elements of V.
6

I definitely wouldn’t recommend option (1).

Option (2) should be perfect if your matrix is in row major order, but if you are working with CUBLAS I presume that isn’t the case? (natural ordering in CUBLAS is the FORTRAN convention).

If you have column major order, then the best approach is probably to have relatively small blocks of threads (say 32 threads per block) read a contiguous segment of V and store to shared memory, then have each thread traverse a row of M and subtract its element from each row entry. That will improve the memory access patterns and should allow coalesced global memory reads and writes. It should also give enough blocks to get reasonable occupancy of the GPU, which is good for hiding latency and keeping the instruction throughput up.

7

It’s indeed in column major order. Thanks for the very helpful and clear answers! Now I can start coding…

8

#Arno can you share the code of subtract a vector from a matrix in cuda.