SM has to finish one block before executing another?

I’m a little confused about this.
Since the SM is the basic execution unit for one block, I’m wondering whether one block holds the whole resource on the SM (execution units, shared memory, etc.).
Can anybody give me an answer please?
Thanks.

I’m a little confused about this.
Since the SM is the basic execution unit for one block, I’m wondering whether one block holds the whole resource on the SM (execution units, shared memory, etc.).
Can anybody give me an answer please?
Thanks.

Multiple blocks can be active on a single SM. A single SM can have 1024 active threads whereas a block can have only 512 threads. The scheduler takes care of switching between the warps and the blocks.

Multiple blocks can be active on a single SM. A single SM can have 1024 active threads whereas a block can have only 512 threads. The scheduler takes care of switching between the warps and the blocks.

What will happen if a block consumes much resource like shared memory (suppose like 80% or more)?

Will the scheduler wait for the block to finish or still have the possibility to switch to other blocks?

What will happen if a block consumes much resource like shared memory (suppose like 80% or more)?

Will the scheduler wait for the block to finish or still have the possibility to switch to other blocks?

the total shared memory (as well as register usage) for the “active blocks” in a SM cannot exceed that available.

so if one block uses 80% or more, and no other require less than 20%, than when that block is “active”, no others can be. there will be no other blocks for the SM to switch to. so in that case the scheduler will have to wait for that block to finish.

if there are no stall evens (such as global memory access) then that won’t be a problem. but any stall events and the SM will be sitting idle on them. thats why you want multiple blocks and warps on an SM. i recommend you check out the “occupancy calculator”. that helps you figure out how many tasks the SM will be able to switch among under different memory constraints, etc.

the total shared memory (as well as register usage) for the “active blocks” in a SM cannot exceed that available.

so if one block uses 80% or more, and no other require less than 20%, than when that block is “active”, no others can be. there will be no other blocks for the SM to switch to. so in that case the scheduler will have to wait for that block to finish.

if there are no stall evens (such as global memory access) then that won’t be a problem. but any stall events and the SM will be sitting idle on them. thats why you want multiple blocks and warps on an SM. i recommend you check out the “occupancy calculator”. that helps you figure out how many tasks the SM will be able to switch among under different memory constraints, etc.

Got it. Thanks for the explanation.

Got it. Thanks for the explanation.