Description
The PTX documentation for .tile::gather4 / .tile::scatter4 says:
These modes work on four rows and hence the instruction will take:
- four tensor coordinates across the dimension 0
- one tensor coordinate across the dimension 1
Reference:
“cp.async.bulk.tensor.2d.shared::cluster.global.tile::gather4…”
" [%0], [%1, {%3, %4, %5, %6, %7}], [%2], %8;"
However, the CUTLASS/CuTe implementation appears to pass one coordinate first, followed by four gathered/scattered coordinates:
SM100_TMA_LOAD_2D_GATHER4::copy(
...,
int32_t const& crd0,
int32_t const& crd1_i0,
int32_t const& crd1_i1,
int32_t const& crd1_i2,
int32_t const& crd1_i3)
The documents on both sides seem to be mismatched; I’m not sure if I’m misunderstanding something.
Thanks for the detailed report and documentation references. To help clarify this discrepancy between the PTX ISA documentation and the CUTLASS/CuTe implementation, we need a bit more context:
-
Have you encountered an actual error or unexpected behavior when using these instructions, or is this purely a documentation discrepancy you noticed?
-
Which source do you believe is correct — the PTX ISA documentation or the CUTLASS/CuTe implementation? Have you tested either approach?
-
Can you provide a minimal code example that demonstrates the issue or clarifies the expected vs. actual behavior?
These details will help us determine whether this is a documentation issue, an implementation deviation, or a misunderstanding of the specification.
Thank you for the clarification. You’re right that this is a documentation discrepancy rather than a functional error. Your reasoning is sound—if the CUTLASS/CuTe implementation were incorrect, it would cause computational errors in production code.
To help route this to the appropriate team for documentation review and clarification, could you please confirm:
- CUDA Toolkit version you’re referencing (e.g., 12.x)
- CUTLASS/CuTe version or commit hash
- Specific PTX ISA documentation version (e.g., PTX ISA 9.3)
This will help ensure the documentation team can verify which source is authoritative and update the documentation accordingly.
Thanks for providing the exact CUDA, CUTLASS, and PTX ISA versions.
This question is about PTX TMA coordinate ordering and the CUTLASS/CuTe implementation rather than TensorRT. I am moving the existing thread to CUDA Programming and Performance, where the CUDA and CUTLASS contributors can review the documentation discrepancy.
You do not need to repost or repeat the version details.
Thanks, Atharva
The PTX manual later states that first the column, then the four rows are specified:
tensorCoords {col_idx, row_idx0, row_idx1, row_idx2, row_idx3}
So the PTX instructions are consistent, just the naming (first vs. second coordinate) is not
Could have something to do with mathematical order: matrices are specified as row, column in math. The row in computer science is y and the column x.
Or with Fortran vs. C convention of arrays