cudnn : 8.0 I found two ways to fuse Convolution and BatchNorm. Use CUDNN_FUSED_SCALE_BIAS_ACTIVATION_CONV_BNSTATS，FusedOpsPlan,cudnnSetFusedOpsConstParamPackAttribute ,cudnnSetFusedOpsVariantParamPackAttribute,etc. I saw that the cudnn api document describes it like this: " As of cuDNN 7.6.0, i…

Regarding the CUDNN_STATUS_BAD_PARAM issue, @hbwx26 can you provide the API log? Developer Guide :: NVIDIA Deep Learning cuDNN Documentation This cuDNN 8.5.0 Developer Guide provides an overview of the NVIDIA cuDNN features such as customizable data layouts, supporting flexible dimensio…

Hello, I didn’t see CUDNN_BACKEND_OPERATION_GEN_STATS_DESCRIPTOR is used in the latest frontend codes. Is Scale_Bias_Activation_ConvBwdFilter pattern already supported by latest CuDNN-Backend? If yes, would recent frontend codes add corresponding sample? Thanks Gino

BTW, normally BatchNorm could be converted to FusedBatchNorm(equal to Scale + Bias) in fine-tune phase, then use Conv_Scale_Bias pattern in inference phase, so why bother to support Scale_Bias_Activation_convolution_genStats pattern? Is it only used in training’s forward phase? Thanks Gino

Fusion of convolution and BatchNorm

AI & Data Science Deep Learning (Training & Inference) cuDNN

spolisetty January 18, 2022, 11:40am 2

Hi,

Please refer the following for cuDNN fuse operator sample.
https://github.com/NVIDIA/cudnn-frontend/blob/main/samples/conv_sample.cpp

For more details,

Thank you.

Topic		Replies	Views
Problem of fusing Convolution and BatchNorm cuDNN	0	669	December 29, 2021
Fuse Operators cuDNN	5	2691	March 31, 2021
Problems encountered in executing cudnnFusedOpsExecute() cuDNN	0	522	September 26, 2022
Cudnn conv+bias fusion using backend cuDNN	1	839	March 31, 2023
Does cudnnFusedOps_t not support backward fusion? cuDNN	1	758	June 4, 2022
Cudnn backend api for fused op cuDNN cudnn	7	2344	July 15, 2021
Cudnn fused conv+bias cuDNN	3	2231	December 9, 2021
How to use cudnn backend to train a cnn network with relu or bn layers? cuDNN	8	1103	July 22, 2021
Get CUDNN_STATUS_BAD_PARAM while executing cudnnFusedOpsExecute() cuDNN debugging-and-troubleshooting	5	1673	October 9, 2022
cuDNN Bug Report: Backend Graph API Conv+Bias Fusion Returns NOT_SUPPORTED cuDNN cuda , cudnn	1	144	November 3, 2025

Fusion of convolution and BatchNorm

Related topics