Can cuda core and tensor core operate at the same time? For example, if my thread block have 8 warps and I write 2 strategies for them. For warp0, it will use cuda core to calculate sth. For the rest 7 warps, they will all use tensor core for some calculation. Can these warps operate in parallel?

Typically yes, you can run them at the same time, each SM Partition has its own tensor core unit and normal cuda cores, so it matters, which warps use one or the other that the workload is distributed well.

For example, if I need to calculate matrix multiplication of 34 * 32 * 32 and I use wmma to calculate 32 * 32 * 32, cuda core for the rest 2 * 32 * 32. Will it faster than by just padding the matrix and then only use wmma?

[image] half-0: Will it faster than by just padding the matrix and then only use wmma? If you spend one hour of your time to set up a quick experiment, you will generate an authoritative answer to this question.

Use cuda core & tensor core at the same time

Accelerated Computing CUDA CUDA Programming and Performance

Robert_Crovella September 9, 2024, 12:32pm 5

Here are some related threads: 1 2 3 4

Topic		Replies	Views
Turing 16x16 MMA, SM usage, 1 or 2? CUDA Programming and Performance	2	1184	December 8, 2018
Can a warp scheduler send instructions to tensor core and cuda core concurrently? CUDA Programming and Performance	2	257	May 5, 2025
About the relationship between warp and tensor_core CUDA Programming and Performance	7	2157	July 7, 2023
How to overlap CUDA core and tensor core computing CUDA Programming and Performance	4	293	June 19, 2025
Is it possible to use cuda core and tensorcore concurrently ? Deep Learning (Training & Inference) mixed-precision	0	1727	October 13, 2019
How to Efficiently Pipeline CUDA Core and Tensor Core Workloads Across Warps for Maximum Throughput? CUDA Programming and Performance	2	299	August 20, 2025
Overlapping CUDA Cores and Tensor Cores CUDA Programming and Performance kernel	2	860	April 7, 2024
Tensor Cores Jetson AGX Xavier	7	1673	May 5, 2020
How to use cuda core and tensor core simultaneously？ GPU-Accelerated Libraries cuda	4	800	August 16, 2022
Mma m8n8k4 on A100 CUDA Programming and Performance	9	920	October 31, 2024

Use cuda core & tensor core at the same time

Related topics