# cublasLtMatmul with leading dimension (lda) \< rows (m)

**URL:** <https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993>\
**Category:** CUDA Programming and Performance\
**Created:** [March 18, 2020, 10:40pm UTC](https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993 "2020-03-18T22:40:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![gary.ballantyne](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@gary.ballantyne](https://forums.developer.nvidia.com/u/gary.ballantyne)\
**Post date:** [March 18, 2020, 10:40pm UTC](https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993/1 "2020-03-18T22:40:50Z")

</div>

Hi

I am studying @mnicely’s code for multiplying two half precision complex matrices ([link](https://github.com/mnicely/cublasLt_examples/blob/master/cublasLt_C16F_TCs.cu)), but cannot generalize it to the case where the leading dimension of A is less than the number of rows of A.

Say matrix A1 is m=3 (rows) x k=3 (columns), and just considering the real part:

1 4 7  
2 5 8  
3 6 9

Then, with column-major format, A1 is stored as an array [1 2 3 4 5 6 7 8 9].

I am interested in the cases like A2 =

1 2 3  
2 3 4  
3 4 5

That is, where each column is offset by less then the number of rows. Here A2 is stored as an array [1 2 3 4 5]. In terms of the GEMM, we have m=3 and lda=1, where lda is the “leading dimesions”, and determines how much we jump through the array for each column ([link](https://devblogs.nvidia.com/cublas-strided-batched-matrix-multiply/)).

When lda\<m, the total number of elements in the array is N = m + (k-1) x lda. For our example, N = 3 + 2\*1 = 5.

So, in @mnicely’s code, I simply change line 339 to:

size\_t sizeA = (k-1)\*lda+m ; //m \* k;

With lda = m (which is the original setting, on line 336), everything if fine (because, sizeA = m\*k in this case). But, when I set lda = 8 (the case I am most interested in), I get the following:

> CUDA error at …/…/…/cublasLt\_C16F\_TCs.cu:279 code=7(CUBLAS\_STATUS\_INVALID\_VALUE) “cublasLtMatmul( ltHandle, operationDesc, alpha, Atransform, AtransformDesc, Btransform, BtransformDesc, beta, Ctransform, CtransformDesc, Ctransform, CtransformDesc, nullptr, workSpace, workSpaceSize, stream )”

Line 279 is the cublasLtMatmul call … but I am having no luck tracking down the source of the error. It seems like the new sizeA should handle things.

The [documentation](https://docs.nvidia.com/cuda/cublas/index.html#cublasLtMatrixLayoutCreate) for cublasLtMatrixLayoutCreate says that the lead dimension must be \>= m … but I think this may be an error (lda\<m certainly works with cutlass::gemm::device::GemmBatched).

Appreciate any suggestions.

Thanks

Gary

---

<div class="post-metadata">

**Author:** ![mnicely](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mnicely/32/14047_2.png) [@mnicely](https://forums.developer.nvidia.com/u/mnicely)\
**Post date:** [March 20, 2020, 7:28pm UTC](https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993/2 "2020-03-20T19:28:32Z")

</div>

Gary,

Your issue may be a bug. I’ll file a bug report for the cublasLt developers to look into it.

Matt

---

<div class="post-metadata">

**Author:** ![gary.ballantyne](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@gary.ballantyne](https://forums.developer.nvidia.com/u/gary.ballantyne)\
**Post date:** [March 20, 2020, 8:54pm UTC](https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993/3 "2020-03-20T20:54:38Z")

</div>

Thanks Matt. Please let me know if I can help with more info, or whatever.

---

<div class="post-metadata">

**Author:** ![gary.ballantyne](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@gary.ballantyne](https://forums.developer.nvidia.com/u/gary.ballantyne)\
**Post date:** [April 8, 2020, 8:24pm UTC](https://forums.developer.nvidia.com/t/cublasltmatmul-with-leading-dimension-lda-rows-m/115993/4 "2020-04-08T20:24:01Z")

</div>

Hi Matt

Is there a bug tracker I can follow?

Cheers

Gary