Per the documentation, cudnnConvolutionBiasActivationForward performs the following operation, where x,y, and z are data pointers:
y = act ( alpha1 * conv(x) + alpha2 * z + bias ).
My question is: is this call expected to always be successful if the operation is performed in-place? That is, can y and z be the same buffer? Further, can x, y, and z be the same buffer?
The documentation for cudnnConvolutionBiasActivationForward does not mention explicitly that in-place operation is supported, though a few other api calls in the documentation are listed explicitly as supporting in-place. I was able to successfully call this function with x = y = z and obtain correct numerical outputs, but I was not certain if this is guaranteed to work every time.
If this call is supported for in-place, in general, how can we tell whether a certain call in the cudnn library supports in-place or not?