# How to Achieve Tighter Kernel Scheduling Across Multiple CUDA Streams? **URL:** \ **Category:** CUDA Programming and Performance\ **Tags:** cudnn, cublas, kernel, cuda\ **Created:** [June 1, 2025, 2:12pm UTC](https://forums.developer.nvidia.com/t/how-to-achieve-tighter-kernel-scheduling-across-multiple-cuda-streams/334941 "2025-06-01T14:12:52Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\ **Post date:** [June 2, 2025, 8:29pm UTC](https://forums.developer.nvidia.com/t/how-to-achieve-tighter-kernel-scheduling-across-multiple-cuda-streams/334941/2 "2025-06-02T20:29:51Z")
> [@lilil111](#): > > **First, how can I make kernels within the same parallel group or across different parallel groups execute with tighter coupling?** I’m looking for techniques to minimize the gaps between kernel executions and achieve more seamless transitions. If it were my code, I would probably start by trying to understand resource utilization of each kernel. If you are actually looking for kernel overlap, that will be a necessary consideration. There are lots of internet posts discussing resource utilization as it relates to concurrent (i.e. overlapping) kernel execution. Beyond that CUDA [graphs](https://developer.nvidia.com/blog/cuda-graphs/) notionally have these ideas in mind. Tighter scheduling, reduced launch overhead, etc. > [@lilil111](#): > > Second, since I’m calling cuDNN and cuBLAS library functions for some operators, these highly optimized libraries generate many short-duration small kernels as observed in nsys. Is there any way to perform kernel fusion on these predefined cuDNN/cuBLAS API functions, or achieve operator fusion at this level? These libraries are closed source and generally “opaque”, meaning they are host code routines that call a set of device level work in some unspecified fashion, with no external control over the sequence. So this is not likely to be a productive avenue, IMO. If you find CUBLAS or CUDNN to be problematic, you should first make sure what you are asking for or expecting is reasonable, then [file a bug](https://forums.developer.nvidia.com/t/how-to-report-a-bug/67911). In the meantime, CUTLASS is open source, giving you more-or-less complete control. I’m not suggesting there is a one-to-one replacement for every CUBLAS/CUDNN function in CUTLASS. They are quite different, but they cover similar areas. > [@lilil111](#): > > **Third, is there any approach to optimize cuDNN and cuBLAS API execution efficiency without relying on runtime profiling?** I would suggest asking detailed CUDNN questions on the [CUDNN forum](https://forums.developer.nvidia.com/c/ai-data-science/deep-learning/cudnn/90), and specific CUBLAS questions on the CUBLAS (math library) [forum](https://forums.developer.nvidia.com/c/accelerated-computing/gpu-accelerated-libraries/12). --- _[View the full topic](https://forums.developer.nvidia.com/t/how-to-achieve-tighter-kernel-scheduling-across-multiple-cuda-streams/334941)._