CUDA 10.1.243 (10.1 update 2) ppc64le run file installation issue

Hello,

I would like to install cuda 10.1.243 as user on ppc64le machine, so using the runfile
The options used for a successful installation on x86 does not work for ppc64le, the result is very awkward: not only it’s not able to install some files, but after installing some stuff it’s removing it …

x86:

[enaultl@ola03 test]$ mkdir -p install-10.1.243; dl/cuda_10.1.243_418.87.00_linux.run --silent --override --toolkit --installpath=$PWD/install-10.1.243
[enaultl@ola03 test]$ echo $?
0
[enaultl@ola03 test]$ ls install-10.1.243/
bin  EULA.txt  include  libnsight  nsight-compute-2019.4.0  nsight-systems-2019.3.7.5  nvvm   src      tools
doc  extras    lib64    libnvvp    nsightee_plugins         nvml                       share  targets  version.txt

ppc64le:

[enaultl@ola02 test]$ mkdir -p install-10.1.243; dl/cuda_10.1.243_418.87.00_linux_ppc64le.run --silent --override --toolkit --installpath=$PWD/install-10.1.243
/products/cuda/test/install-10.1.243/include/cublas.h can't be opened
/products/cuda/test/install-10.1.243/include/cublasLt.h can't be opened
/products/cuda/test/install-10.1.243/include/cublasXt.h can't be opened
/products/cuda/test/install-10.1.243/include/cublas_api.h can't be opened
/products/cuda/test/install-10.1.243/include/cublas_v2.h can't be opened
/products/cuda/test/install-10.1.243/include/nvblas.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran.c can't be opened
/products/cuda/test/install-10.1.243/src/fortran.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran_common.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran_thunking.c can't be opened
/products/cuda/test/install-10.1.243/src/fortran_thunking.h can't be opened
 Installation failed. See log at /tmp/cuda-installer.log for details.
[enaultl@ola02 test]$ echo $?
1
[enaultl@ola02 test]$ ls install-10.1.243/
targets
[enaultl@ola02 test]$ grep ^.ERROR /tmp/cuda-installer.log
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas.h", "/products/cuda/test/install-10.1.243/include/cublas.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublasLt.h", "/products/cuda/test/install-10.1.243/include/cublasLt.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublasXt.h", "/products/cuda/test/install-10.1.243/include/cublasXt.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas_api.h", "/products/cuda/test/install-10.1.243/include/cublas_api.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas_v2.h", "/products/cuda/test/install-10.1.243/include/cublas_v2.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/nvblas.h", "/products/cuda/test/install-10.1.243/include/nvblas.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran.c", "/products/cuda/test/install-10.1.243/src/fortran.c"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran.h", "/products/cuda/test/install-10.1.243/src/fortran.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_common.h", "/products/cuda/test/install-10.1.243/src/fortran_common.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_thunking.c", "/products/cuda/test/install-10.1.243/src/fortran_thunking.c"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_thunking.h", "/products/cuda/test/install-10.1.243/src/fortran_thunking.h"

Also, after installing some files, it’s removing them:

...
[INFO]: Installed: /products/cuda/test/install-10.1.243/doc/pdf/Nsight_Eclipse_Plugins_Getting_Started.pdf
[INFO]: Installed: /products/cuda/test/install-10.1.243/tools/CUDA_Occupancy_Calculator.xls
[INFO]: Skipping copy. File already exists at: /products/cuda/test/install-10.1.243/version.txt
[WARNING]: Unable to write to directory: /usr/share/man
[WARNING]: Cannot write to manpage directory: /usr/share/man
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/EULA.txt
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/bin/cuda-uninstaller
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/version.txt
...

I tried to set also set --defaultroot and some other, but I didn’t find any solution.
How to solve the issue ?

Hi Ludo;

I have filed internal bug 2715295 for this issue.
You should have visibility to the bug soon.

When will the new version be released?Thank u.

Same issue here, any news about the fix?

CUDA 10.2 was just released.

Just tried 10.2 and works perfectly, thanks!

Thank you Bob!
It’s working fine with CUDA 10.2.89.

For versions prior to 10.2.89, I have been provided an other work-around, I am not sure if I can put it here.

Thanks again,
Ludo

I’d be very interested in a work-around for CUDA 10.1.243 because one is needed for e.g. EasyBuild, see https://github.com/easybuilders/easybuild-easyconfigs/issues/9437

What is going on with this? I want 10.1.243 via runfile but am running into the same issues described above.

This is the workaround recipe for 10.1.243 runfile installer:

$ mkdir -p `pwd`/install-10.1.243/targets/x86_64-linux/include `pwd`/install-10.1.243/src

$ ln -s targets/x86_64-linux/include/ `pwd`/install-10.1.243/include

$ sh cuda_10.1.243_418.87.00_linux_ppc64le.run --silent --override --toolkit --installpath=`pwd`/install-10.1.243

However we recommend users move forward to 10.2

Thank you Robert! On line 3 of your response, is there supposed to be anything before “targets/x86_64-linux/include/” ?
Also, will this work for ppc64le if I just change the x86_64-linux to ppc64le-linux ? thanks

That exact recipe is what was tested. It was confirmed working verbatim by 2 independent parties. I haven’t tested it myself.