How to activate with softplus in C backend?

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

Hello,

I am using cudnnActivationMode_t enum for sigmoid, relu, tanh activation functions as my C++ class member m_activationMode to set member m_activationDescriptor (which is used for subsequent cudnnActivationForward call for inferencing):

cudnnSetActivationDescriptor (m_activationDescriptor, m_activationMode, CUDNN_NOT_PROPAGATE_NAN, alpha));

Unfortunately, I see no CUDNN_ACTIVATION_SOFTPLUS kind under 9.5/include/cudnn_graph_v9.h for enum cudnnActivationMode_t.

I find softplus in above header, but it is in cudnnPointwiseMode_t and cudnnBackendAttributeName_t enums. The former enum is not used anywhere else in the library. There is hope since latter enum is used in cudnnBackendSetAttribute but not sure how to correctly use that function.

Could you please point to or provide an example for softplus activation?

Thank you

2

Hi @westwood ,
Cheking on this and will get back,.

Thanks

1 Like