Tensorflow with CuDNN without building from source

If Tensorflow is installed with pip will it be able to accelerate with CuDNN (if available?) how can I tell if this is happening? Or do I have to build TF from source in order to benefit from CuDNN?

TF-gpu from pip will use cudnn

You can tell it is using cudnn by deleting the library and watch TF-gpu throw errors.