Batched multiplication with sparse matrices and dense vectors

[ben.pollock]

I have a situation where I need to multiply various sparse matrices with an unchanging dense vector. I first attempted to solve this using cusparseSpSV and cusparseCooSetStridedBatch, however, this resulted in an error when calling cusparseSpSV_bufferSize that said “On entry to cusparseSpMV_bufferSize(): batched computation is unsupported”.

Then, I tried cusparseSpMM. This worked when the dense matrix B had more than one column, but when I tried to represent my dense vector as a dense matrix with a single column, I encountered some strange behaviour. The multiplication would work correctly for the first matrix-vector product, but all subsequent matrix-vector products would have all zeros in the resultant C vector. Looking at the docs for cuSPARSE I’m fairly confident I’ve found why this is happening. Under cusparseSpMM performance notes, there is a dot point that says “For n == 1, the routine uses cusparseSpMV() as fallback”, where n is the number of columns in the B matrix. So, I’m guessing what happens is that cusparseSpMM recognises that B only has 1 column, and uses cusparseSpMV instead, but cusparseSpMV can’t do batched operations and so only performs the first matrix-vector product.

So, I was wondering if there are any plans to include batched operations for cusparseSpMV, or a way to disable the fallback to cusparseSpMV in cusparseSpMM, or another workaround? Thanks.

[qanhpham]

Hi. We haven’t had plan to do batched SpMV, but if there are many requests we’ll consider doing it.
2 potential workarounds that I can think of:

1. pad the vector to make it a 2-column matrix to bypass the SpMV fallback.
2. Concatenate your matrices into a single matrix of size (k*m) x n, where k is the batch size. This requires a preprocessing step for constructing the big matrix.

Would you mind telling us what is your use case of batched SpMV?

[ben.pollock]

Hi thanks for the response. You’re right, batched SpMV isn’t really necessary in this case, since my dense vector is unchanging I can concatenate my matrices into one large matrix and multiply that with the dense vector. Thanks.

[qanhpham]

Glad it helps. Just one note that when you concatenate the matrices you’ll need to update row indexes of subsequent matrices so that the indexes are in range [0, k*m).