Truncated loop-exit compare in PTX prevents ptxas from unrolling loops

Accelerated Computing CUDA CUDA Programming and Performance
1

While investigating a 3x slowdown in a matmul kernel, we traced it to a single instruction pattern in the PTX loop latch. Unrolling is not happening in ptxas when loop exit compare is done on result of truncate op.

add.s64      %rd1, %rd1, 32768;
cvt.u32.u64  %r2, %rd1;          // truncate to 32 bits
setp.ne.b32  %p1, %r2, 524288;   // compare the truncated copy
@%p1 bra     LOOP;

However, for the below pattern unrolling is still happening

add.s32     %r1, %r1, 16384;
setp.ne.s32 %p1, %r1, 262144;   // IV compared directly → trip count provable
@%p1 bra    LOOP;

The problematic pattern was produced by LLVM’s Loop Strength Reduction (newer LLVM versions rewrite the exit compare onto a truncated 64-bit address IV; older LLVM kept it on the 32-bit counter). Editing the PTX to compare the IV directly — either setp.ne.s64 on the 64-bit register or setp on the 32-bit IV — immediately restores ptxas unrolling.

ptx.txt (8.3 KB)

2 Likes