Clarify dimension ordering for SM100 TMA .tile::gather4/.tile::scatter4 coordinates

Description

The PTX documentation for .tile::gather4 / .tile::scatter4 says:

These modes work on four rows and hence the instruction will take:

  1. four tensor coordinates across the dimension 0
  2. one tensor coordinate across the dimension 1

Reference:

“cp.async.bulk.tensor.2d.shared::cluster.global.tile::gather4…”
" [%0], [%1, {%3, %4, %5, %6, %7}], [%2], %8;"

However, the CUTLASS/CuTe implementation appears to pass one coordinate first, followed by four gathered/scattered coordinates:

SM100_TMA_LOAD_2D_GATHER4::copy(
    ...,
    int32_t const& crd0,
    int32_t const& crd1_i0,
    int32_t const& crd1_i1,
    int32_t const& crd1_i2,
    int32_t const& crd1_i3)

The documents on both sides seem to be mismatched; I’m not sure if I’m misunderstanding something.

Thanks for the detailed report and documentation references. To help clarify this discrepancy between the PTX ISA documentation and the CUTLASS/CuTe implementation, we need a bit more context:

  1. Have you encountered an actual error or unexpected behavior when using these instructions, or is this purely a documentation discrepancy you noticed?

  2. Which source do you believe is correct — the PTX ISA documentation or the CUTLASS/CuTe implementation? Have you tested either approach?

  3. Can you provide a minimal code example that demonstrates the issue or clarifies the expected vs. actual behavior?

These details will help us determine whether this is a documentation issue, an implementation deviation, or a misunderstanding of the specification.

Hi,

  1. This is purely a documentation discrepancy I noticed.

  2. I believe the implementation in CUTLASS/CuTe is correct; otherwise, using CUTLASS/CuTe would lead to serious computational errors. Furthermore, according to the logic in the PTX documentation, the .tile::gather4 / .tile::scatter4 modes only exist when the tensor is 2D, so the second dimension should have four numbers, i.e., dimension 1.

  3. I haven’t verified this experimentally myself; I’ve only read the documentation.

Thanks

Thank you for the clarification. You’re right that this is a documentation discrepancy rather than a functional error. Your reasoning is sound—if the CUTLASS/CuTe implementation were incorrect, it would cause computational errors in production code.

To help route this to the appropriate team for documentation review and clarification, could you please confirm:

  1. CUDA Toolkit version you’re referencing (e.g., 12.x)
  2. CUTLASS/CuTe version or commit hash
  3. Specific PTX ISA documentation version (e.g., PTX ISA 9.3)

This will help ensure the documentation team can verify which source is authoritative and update the documentation accordingly.

Hi,

  1. cuda 12.8
  2. cutlass commit 5c54bee12b1efc83fda51bbfb116b2f0a66b4591
  3. PTX ISA 9.3

Thanks.

Thanks for providing the exact CUDA, CUTLASS, and PTX ISA versions.

This question is about PTX TMA coordinate ordering and the CUTLASS/CuTe implementation rather than TensorRT. I am moving the existing thread to CUDA Programming and Performance, where the CUDA and CUTLASS contributors can review the documentation discrepancy.

You do not need to repost or repeat the version details.

Thanks, Atharva

The PTX manual later states that first the column, then the four rows are specified:

tensorCoords {col_idx, row_idx0, row_idx1, row_idx2, row_idx3}

So the PTX instructions are consistent, just the naming (first vs. second coordinate) is not

Could have something to do with mathematical order: matrices are specified as row, column in math. The row in computer science is y and the column x.

Or with Fortran vs. C convention of arrays