Better Kernel Performance When Using More Global Memory Reads - Help Needed With Analysis

[Simon_H]

@Greg Thank you for your explanation!

@Greg

The Streaming Multiprocessor (SM) in consumer GPUs (not 100 class data center) has significantly lower issue rate for double precision (2 threads/clk/SM). The double precision unit is shared by all 4 SM sub-partitions and the issue and return path uses the same path as the texture unit.

I see. Is such information written down anywhere publicly available or does this kind of information only come through answers in this forum, SO etc. by nvidia employees? I found this turing whitepaper but it seems to not describe the TU117 chip, which is used by the Quadro T2000 and especially not in such detail.

@Greg

stalled_tex_throttle is due to the long sequence of FP64 instructions (DMUL, DFMA, DADD) at the end of the kernel.

In the coeffs_as_constant_memory the FP64 pipe is not used as well as coeffs_as_global_buffer. This is due to the compiler grouping all memory then all FP64 operations. This leads to the FP64 pipe not being used for a period of time then used at 100%. For coeffs_as_global_buffer the compiler interleaves memory loads and FP64 resulting in sustaining a higher FP64 pipe utilization. The kernels is not memory throughput limited so the additional loads for the coefficients does not hurt performance.

Oh I understand. Hm, I tried to reorder the PTX by hand, interleaving the DFMA and LDG instruction near the end (the PTX is stored when using OpenCL’s feature to store as binary file and can be used in a subsequent run), but with no change in runtime. Unfortunately I cannot analyze that in ncu using pOCL, as then the stored binary file seems to really be in binary format instead of PTX. However I was able to retrieve the SASS code by calling:

nvcc -arch=sm_75 residualFixedKernelVersions.ptx -dlink
cuobjdump -sass a_dlink.o

It reveals that the compiler seems to ignore my reordering attempts, still listing all the DFMA at the end (I also read similar information anywhere before, but wanted to try myself). So I guess there is no feasible way to manually reorder the SASS instructions (as already thoroughly discussed here). Maybe, based one of your(?) answers on SO, I might be able to insert dependent instructions, preventing ptxas to reorder things. Maybe I’ll have a look into that… And I’ll try to get access to a cluster GPU to try out the kernels!

Again, thank you very much!