In CuDNN 7.1 (at least on Windows), cudnnConvolutionBiasActivationForward does not operate as described in the documentation.
Instead of using the activation descriptor actually provided by the user (according to the docs, it should support a choice of two activation modes - RELU and identity), it seems to apply RELU all the time, even if you set the activation descriptor to identity. It does pay attention to that descriptor - it’ll throw up an error if you attempt to pass an unsupported activation mode - but, as long as a legal descriptor is passed, its return values are always nonnegative, as you’d expect after RELU.