problem in build-install tensorflow

ERROR: /home/burkro/.cache/bazel/_bazel_burkro/225de5e6309bfb479c3acc4f8af259bc/external/nccl_archive/BUILD.bazel:139:1: Executing genrule @nccl_archive//:device_code_fatbin_h failed (Exit 1)
fatbinary fatal : Unknown option ‘-bin2c-path’
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 623.452s, Critical Path: 88.04s
INFO: 1372 processes: 1372 local.
FAILED: Build did NOT complete successfully

I have tied cuda10.0,10.1,10.2 with cudnn 7.6.5, and also tied tensorflow 1.13.2,1.14.0,1.15.0,2.0.0, all version has this problem and build failed. What’s the problem and how to resolve it?

I use --verbose_failure get more detail:

ERROR: /home/burkro/.cache/bazel/_bazel_burkro/225de5e6309bfb479c3acc4f8af259bc/external/nccl_archive/BUILD.bazel:139:1: Executing genrule @nccl_archive//:device_code_fatbin_h failed (Exit 1): bash failed: error executing command
(cd /home/burkro/.cache/bazel/_bazel_burkro/225de5e6309bfb479c3acc4f8af259bc/execroot/org_tensorflow &&
exec env -
/bin/bash -c ‘source external/bazel_tools/tools/genrule/; external/local_config_nccl/cuda/bin/fatbinary -64 --cmdline=–compile-only --link --bin2c-path $(dirname external/local_config_nccl/cuda/bin/bin2c) --compress-all --image=profile=sm_61,file=bazel-out/host/genfiles/external/nccl_archive/device_code_sm_61.cubin --create=%{name}.fatbin --embedded-fatbin=bazel-out/host/genfiles/external/nccl_archive/device_code.fatbin.h’)
Execution platform: @bazel_tools//platforms:host_platform
fatbinary fatal : Unknown option ‘-bin2c-path’

I’m surprised they let you post that. I posted a problem with CuDNN failing to allocate memory while using tensorflow and they didnt even let that get posted…

the problem was with a bog standard installation of everything so its sort of a BIG problem

What big problem? Acctually one year ago, I build-install tensorflow1.13.0 with cuda10.0 on 18.04.2, today I just only re-install the ubuntu to 18.04.3, the kernel from 4.15 up to 5.0. So failed. :(.
I just want to know the problem is cuda, cudnn, or nccl? and how to resolve it?
Do you mean cudnn has big problem?

It seems I find the key reason. That is CUDNN. The latest version maybe has not supported by tensorflow. After I down-grade to v7.6.2, build successfully.