CUDA 10.1.243 (10.1 update 2) ppc64le run file installation issue

Accelerated Computing CUDA CUDA Setup and Installation
1

Hello,

I would like to install cuda 10.1.243 as user on ppc64le machine, so using the runfile
The options used for a successful installation on x86 does not work for ppc64le, the result is very awkward: not only it’s not able to install some files, but after installing some stuff it’s removing it …

x86:

[enaultl@ola03 test]$ mkdir -p install-10.1.243; dl/cuda_10.1.243_418.87.00_linux.run --silent --override --toolkit --installpath=$PWD/install-10.1.243
[enaultl@ola03 test]$ echo $?
0
[enaultl@ola03 test]$ ls install-10.1.243/
bin  EULA.txt  include  libnsight  nsight-compute-2019.4.0  nsight-systems-2019.3.7.5  nvvm   src      tools
doc  extras    lib64    libnvvp    nsightee_plugins         nvml                       share  targets  version.txt

ppc64le:

[enaultl@ola02 test]$ mkdir -p install-10.1.243; dl/cuda_10.1.243_418.87.00_linux_ppc64le.run --silent --override --toolkit --installpath=$PWD/install-10.1.243
/products/cuda/test/install-10.1.243/include/cublas.h can't be opened
/products/cuda/test/install-10.1.243/include/cublasLt.h can't be opened
/products/cuda/test/install-10.1.243/include/cublasXt.h can't be opened
/products/cuda/test/install-10.1.243/include/cublas_api.h can't be opened
/products/cuda/test/install-10.1.243/include/cublas_v2.h can't be opened
/products/cuda/test/install-10.1.243/include/nvblas.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran.c can't be opened
/products/cuda/test/install-10.1.243/src/fortran.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran_common.h can't be opened
/products/cuda/test/install-10.1.243/src/fortran_thunking.c can't be opened
/products/cuda/test/install-10.1.243/src/fortran_thunking.h can't be opened
 Installation failed. See log at /tmp/cuda-installer.log for details.
[enaultl@ola02 test]$ echo $?
1
[enaultl@ola02 test]$ ls install-10.1.243/
targets
[enaultl@ola02 test]$ grep ^.ERROR /tmp/cuda-installer.log
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas.h", "/products/cuda/test/install-10.1.243/include/cublas.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublasLt.h", "/products/cuda/test/install-10.1.243/include/cublasLt.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublasXt.h", "/products/cuda/test/install-10.1.243/include/cublasXt.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas_api.h", "/products/cuda/test/install-10.1.243/include/cublas_api.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/cublas_v2.h", "/products/cuda/test/install-10.1.243/include/cublas_v2.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/include/nvblas.h", "/products/cuda/test/install-10.1.243/include/nvblas.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran.c", "/products/cuda/test/install-10.1.243/src/fortran.c"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran.h", "/products/cuda/test/install-10.1.243/src/fortran.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_common.h", "/products/cuda/test/install-10.1.243/src/fortran_common.h"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_thunking.c", "/products/cuda/test/install-10.1.243/src/fortran_thunking.c"
[ERROR]: boost::filesystem::copy_file: No such file or directory: "./builds/cublas/src/fortran_thunking.h", "/products/cuda/test/install-10.1.243/src/fortran_thunking.h"

Also, after installing some files, it’s removing them:

...
[INFO]: Installed: /products/cuda/test/install-10.1.243/doc/pdf/Nsight_Eclipse_Plugins_Getting_Started.pdf
[INFO]: Installed: /products/cuda/test/install-10.1.243/tools/CUDA_Occupancy_Calculator.xls
[INFO]: Skipping copy. File already exists at: /products/cuda/test/install-10.1.243/version.txt
[WARNING]: Unable to write to directory: /usr/share/man
[WARNING]: Cannot write to manpage directory: /usr/share/man
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/EULA.txt
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/bin/cuda-uninstaller
[INFO]: md5 matches, removing file /products/cuda/test/install-10.1.243/version.txt
...

I tried to set also set --defaultroot and some other, but I didn’t find any solution.
How to solve the issue ?

2

Hi Ludo;

I have filed internal bug 2715295 for this issue.
You should have visibility to the bug soon.

3

When will the new version be released?Thank u.

4

Same issue here, any news about the fix?

5

CUDA 10.2 was just released.

6

Just tried 10.2 and works perfectly, thanks!

7

Thank you Bob!
It’s working fine with CUDA 10.2.89.

For versions prior to 10.2.89, I have been provided an other work-around, I am not sure if I can put it here.

Thanks again,
Ludo

8

I’d be very interested in a work-around for CUDA 10.1.243 because one is needed for e.g. EasyBuild, see CUDA 10.1.243 install broken on power9 · Issue #9437 · easybuilders/easybuild-easyconfigs · GitHub

9

What is going on with this? I want 10.1.243 via runfile but am running into the same issues described above.

10

This is the workaround recipe for 10.1.243 runfile installer:

$ mkdir -p `pwd`/install-10.1.243/targets/x86_64-linux/include `pwd`/install-10.1.243/src

$ ln -s targets/x86_64-linux/include/ `pwd`/install-10.1.243/include

$ sh cuda_10.1.243_418.87.00_linux_ppc64le.run --silent --override --toolkit --installpath=`pwd`/install-10.1.243

However we recommend users move forward to 10.2

11

Thank you Robert! On line 3 of your response, is there supposed to be anything before “targets/x86_64-linux/include/” ?
Also, will this work for ppc64le if I just change the x86_64-linux to ppc64le-linux ? thanks

12

That exact recipe is what was tested. It was confirmed working verbatim by 2 independent parties. I haven’t tested it myself.