Hello devtalk community,

I’m sorry if there are any similar topics to the one I post, but I couldn’t find any and decided to ask for your help directly.

The cuDNN documentation says that in the following function

```
cudnnSetActivationDescriptor(
cudnnActivationDescriptor_t activationDesc,
cudnnActivationMode_t mode,
cudnnNanPropagation_t reluNanOpt,
double coef)
```

the parameter **coef** specifies the alpha coefficient when the activation mode is set to **CUDNN_ACTIVATION_RELU**. In this way we can specify a Leaky ReLU activation. So by choosing **0 < coeff < 1** the activation should be - **f(x) = coeff * x for x < 0, f(x) = x for x >= 0**.

When I call

```
cudnnSetActivationDescriptor(relu_desc, CUDNN_ACTIVATION_RELU, CUDNN_NOT_PROPAGATE_NAN, 0.1)
cudnnActivationForward(cudnn, relu_desc, &alpha, src_tensor, dev_src, &beta, relu_tensor, dev_relu)
```

with

```
alpha = 1; beta = 0;
```

**x > 0 is mapped to x** and **x < 0 to 0** instead of **x < 0 to x * 0.1**. So the combination **CUDNN_ACTIVATION_RELU** and **0 < coeff < 1** is not working like a Leaky ReLU.

Is this the normal behaviour of the function? If yes, what combination is supposed to be Leaky ReLU?

I’m using **cuda compilation tools, release 10.1, V10.1.168** and **cudnn-10.1-windows10-x64-v7.6.0.64**

Thank you for your time!