--ptxas-options=-v question

–ptxas-options=-v gives me:
Used 52 registers, 56+28 bytes lmem, 12448+12432 bytes smem, 96 bytes cmem[0], 124 bytes cmem[1]

What do 56+28 bytes lmem and 12448+12432 bytes smem mean ?
And cmem[0] and [1] ?

I could not find it in the docs.

lmem is local memory. It is used either because the compiler spilled registers to local memory, or because you declared a local array in the kernel code. Only very small local arrays (4 elements or fewer, I think) that are not indexed with a variable (since we don’t have an indexable register file) are placed in registers.

smem is shared memory. This is used both for data you declare shared and for global function parameters, as well as shared parameters like blockIdx, blockDim, and gridDim. (threadIdx starts out in register 0, since it is a per-thread value). This smem count doesn’t include any dynamically allocated shared memory (“extern shared float foo”).

cmem[0] and cmem[1] are two different constant memory banks. If you use constant literal values in your code these are sometimes placed in cmem, and if you declare any constant data it is placed there.

I’m not sure what the a+b notation means, I will ask.


OK, the + notation is used to refer to “user” + “reserved” usage. So for example in the case of shared memory, “user” is user defined shared variables, while “reserved” would be configuration data such as blockDim and gridDim, and formal parameters to kernels.

Having said that, there’s currently a bug in ptxas that is causing incorrect values to the right of the “+” sign, so please ignore those for now.

Thanks for reporting this.


Okay, happy it is a bug, as I could not really relate all the values. I hope the output will become something like (RFE follows ;)) :

A (B+C) meaning A = total amount, B = user defined, C = input parameters, blockDim, etc. That way it is easier to see the value to be used in the occupancy calculator.