I know that the volta architecture can run INT32 instructions and FP32 instructions concurrently. But I don’t know about the INT 32 and FP64 pair.
I want to do polynomial multiplications in parallel, so I want to do FFT and NTT in parallel to use the GPU’s resource as much as possible.
More precisely, I’m using INT64 operations in NTT, but, as far as I know, INT64 operations are executed as combinations of INT32 instructions.
GV100 has several math pipelines:
FMA pipe - executes FP32 instructions and IMAD (integer multiply and add)
ALU pipe - executes INT32 (not IMAD), logical operations, binary operations, and data movement operations
FP64 pipe - executes FP64 instructions
FP16 pipe - executes FP16x2 instructions
Tensor pipe - executes matrix multiply and accumulate instructions
The FP64, FP16, and Tensor pipe use the same dispatch port so you cannot dispatch to these pipes at the same time.
The FMA and ALU pipeline each have separate dispatch ports. It takes 2 cycles to dispatch a warp to the each of these pipes (16 cores).
Concurrent execution is done by alternating instruction dispatch to different pipes.
On GV100 INT64 math is implemented by various units including:
FMA pipe - IMAD instruction
ALU pipe - LEA instruction
The answer to your question is not straight-forward.
Pipeline utilization metrics for GV100 are available in CUDA >= 10.1 tools. Nsight 2019.1 adds the pipeline utilization in the Compute Workload Analysis section.
According to the A100 whitepaper, each sub-SM (1/4 SM) has16 FP32, 16 INT32, and 8 FP64. Image there are 4 cycles: “cycle 1” and “cycle 3” are used to issue FP32 instructions (32 threads splitted into 2 round on FP32, so each issue can saturate FP32 for 2 cycles, and the issue interval should be 2.), “cycle 2” is used to issue FP64 instructions (32 threads splitted into 4 round), and there leaves only “cycle 4” to issue INT32 instructions. However, it takes 2 cycles out of 4 to achieve INT32 peak performance, because each issue on INT32 last 2 cycles (also because 32 threads splitted on 16 INT32). This means INT32 can not achieve peak performance.
#FP32 is 32 per 1/4 SM, which means every cycle must be used to issue FP32 instructions to achieve FP32 peak performance, and there is no rest issue opportunity for FP64 and INT32.
How these pipelines run in parallel with peak performance? Do I have a misunderstanding of how it works?
In my response, I stated that these instructions can be executed concurrently. I’m not aware of any claims by NVIDIA that a GPU can simultaneously achieve peak performance simultaneously in every possible pipeline simultaneously.
