I’m getting the same missing cublas error despite following tensorflow’s instructions exactly.
Finally managed to solve this thanks to matthew.smith3’s comment.
I personally didn’t have any of the cublas files in /usr/lib/x86_64-linux-gnu/. So after extracting the .run file I just dumped the contents of the cuda-toolkit folder into /usr/local/cuda-10.1/lib64.
Then I created a symbolic link for each of the cublas files (including cublasLt) to /usr/lib/x86_64-linux-gnu/.
sudo ln -s /usr/local/cuda-10.1/lib64/libcublas.so /usr/lib/x86_64-linux-gnu/libcublas.so
sudo ln -s /usr/local/cuda-10.1/lib64/libcublas.so.10 /usr/lib/x86_64-linux-gnu/libcublas.so.10
sudo ln -s /usr/local/cuda-10.1/lib64/libcublas.so.10.1 /usr/lib/x86_64-linux-gnu/libcublas.so.10.1
etc.