# Wmma load\_mma\_sync API

**URL:** <https://forums.developer.nvidia.com/t/wmma-load-mma-sync-api/265654>\
**Category:** CUDA Programming and Performance\
**Created:** [September 7, 2023, 7:49am UTC](https://forums.developer.nvidia.com/t/wmma-load-mma-sync-api/265654 "2023-09-07T07:49:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jeromeku](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@jeromeku](https://forums.developer.nvidia.com/u/jeromeku)\
**Post date:** [September 7, 2023, 7:49am UTC](https://forums.developer.nvidia.com/t/wmma-load-mma-sync-api/265654/1 "2023-09-07T07:49:36Z")

</div>

Following the example from [Programming Tensor Cores](https://developer.nvidia.com/blog/programming-tensor-cores-cuda-9/), code [here](https://github.com/NVIDIA-developer-blog/code-samples/blob/master/posts/tensor-cores/simpleTensorCoreGEMM.cu), if I lay out A and B **row-major order** and run `wmma_example` (`alpha` and `beta` set to 1 for simplicity), I get the right answer for C. However, the kernel specifies A and B as [col-major](https://github.com/NVIDIA-developer-blog/code-samples/blob/3a495ce6219629f819c51cdaed5cd7bf5d91a636/posts/tensor-cores/simpleTensorCoreGEMM.cu#L86C2-L86C2) and also indexes A and B in column-major order [here](https://github.com/NVIDIA-developer-blog/code-samples/blob/3a495ce6219629f819c51cdaed5cd7bf5d91a636/posts/tensor-cores/simpleTensorCoreGEMM.cu#L104).

Also, the leading dimension for A is M and for B is K (`lda` and `ldb`), which the documentation states is the stride length between columns for `col-major` layouts is confusing, as stride-length is 1 between columns for column major by definition?

What am I missing?

---

<div class="post-metadata">

**Author:** ![njuffa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@njuffa](https://forums.developer.nvidia.com/u/njuffa)\
**Post date:** [September 7, 2023, 8:28am UTC](https://forums.developer.nvidia.com/t/wmma-load-mma-sync-api/265654/2 "2023-09-07T08:28:20Z")

</div>

> [@jeromeku](#):
>
> as stride-length is 1 between columns for column major by definition

The leading dimension in column-major storage is the spacing, in elements, between the first element of column N and the first element of column N+1. Unless the matrix is a 1x1 matrix, it is therefore generally greater than 1.

Note that in BLAS primitives involving matrices (of which `GEMM` is an important one) leading dimensions are generally used to allow for the easy processing of sub-matrices contained within a larger matrix. In which case, for column-major storage, the spacing between columns of the sub-matrix is based on the column “height” of the larger matrix it is a part of, which is the leading dimension.