Understanding PTXAS output

Accelerated Computing CUDA CUDA Programming and Performance
1

I had some questions about the register usage/spill information as reported in ptxas info. Lets consider the below example below compiled using cuda/12.2 on A100.

ptxas info    : Function properties for _ZN4pele7physics9reactions5utils20fKernelSpecBase_CUDAINS2_7CYOrderEEEvidPKdPdS6_S6_S6_
    1136 bytes stack frame, 1808 bytes spill stores, 2280 bytes spill loads
ptxas info    : Used 254 registers, 1136 bytes cumulative stack size, 408 bytes cmem[0], 16576 bytes cmem[2]
  1. What does 254 register correspond to? Is there a notion of scalar vs vector registers, or does the compiler not distinguish between the two?
  2. Following 1, are the spills reported corresponding to scalar or vector registers or combined?
  3. Is there a notion of scalar registers spilling into vector registers, or do both spill into the stack with no intermediate spill locations?
  4. What is the difference between the stack frame and cumulative stack size ? Does the former correspond the use of stack for register spills, and later is the total stack usage?
2
  1. Scalar registers and vector registers (PTX name for types like int4 or short2) are stored in the same hardware registers (register file).
    The hardware register size is 32 bits (4 bytes). Longer scalar or vector registers are stored in several consecutive hardware registers.
  2. Combined.
    3.+4. I am not 100% sure about details and nomenclature. There is local memory with fixed storage locations, there is stack with LIFO storage. The stack could also be used for calling non-inlined or recursive functions. Perhaps that is the reason that two values are given.
    Local memory is also used (in addition to spilling registers, when too many are needed), if you access local (in the C sense)=automatic=scoped arrays with a dynamic (= known only at runtime) index.
3

Spilling is something that happens at SASS (machine code) level, based on decisions made by ptxas, the optimizing PTX to SASS compiler. At the SASS level there is no notion of vector registers, just a bunch of 32-bit general purpose registers. A data type that needs more than 32 bits of storage, such as double, occupies multiple of these registers. While some instructions require objects of wide data types to occupy naturally-aligned register groups, in general the compiler can and does use non-consecutive and/or non-aligned groups of registers where such requirements are not in force.

That being said, vectorized data transfers wider than 32 bits can be observed during spilling and subsequent restoration, but best I recall (fairly vaguely at this point!) from looking at examples of this, this is based on efficiency considerations and not strictly dependent on data type.

In SASS there are also some special registers, in particular for predicates. I am reasonably sure that I have observed predicates being spilled to GPRs using the P2R instruction on a few occasions, to be restored later with the R2P instruction. As this is a special kind of spilling that does not spill to memory I doubt it shows up in ptxas compilation statistics.