Hi! I am learning persistant L2, so if we set a data AA in global as persistant, the hardware will load it to L2 in advance and keep it persistant. But when we write back data to AA, will the data still keep persistant in L2?
I don’t think so. The L2 does not get populated until you actually request data (or write data).
1 Like
OK, OK. Actually what I really want to ask is, when we write the data back to global memory, will it firstly be written into our persistant L2? I mean, will this data be persistant, or this specific L2 memory chunk be persistant?
If the address you are writing to is within the persistent defined region, it should obey those rules. If not, it won’t. The behavior here should be the same as other L2 behavior. All writes or reads targetting global space allocated on the device will flow through the L2. That means both reads and writes will attempt to hit in L2. Nothing bypasses the L2. And of course, apply the additional rules imposed by your L2 persistence setup.
I know in A100, the L2 will not be flushed after a kernel. So I guess using persistant L2 will be the same? After one kernel, the L2 will still be persistantly stayed there for next kernel, unless manually delete this persistancy?
Thank you!!!
I used persistant cache for matmul, I stored all inputs in L2 but the L2 hit rate is not 100%…Maybe because it is too large, and the first read is always a miss?
So I guess, for matmul algorithm, persistant L2 is not that useful? Because the data reuse is naturally few…
Another problem is, initially the data are stored on global memory, but when I run the algorithm, I want to set some data as persistant, like some weight matrix, and after using them, some other data need to become persistant, but really move data in global memory is very slow…Can I just use pointer to set here as persistant, and then there as persistant within the kernel?
Thanks!