Install Onnxruntime with JetPack 4.4 on AGX

Description

Dear all,

I wanna install the Onnxruntime-gpu version on AGX that I tried to build from source. I already succeeded it with JetPack 4.3 version on AGX .

I upgraded to JetPack v4.4 on AGX; however, I cannot build onnxruntime-gpu version successfully.
I can build the .whl file for onnxruntime cpu version and install this .whl without any error. (Command: ./build.sh)

I want to install gpu/tensorrt version of onnxruntime. I got error below.

Command:

$ ./build.sh --config Release --update --build --build_wheel --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu --tensorrt_home /usr/lib/aarch64-linux-gnu

Output:

2020-06-08 14:41:19,434 Build [INFO] - Build started
2020-06-08 14:41:19,436 Build [DEBUG] - Running subprocess in '/home/nvidia/onnxruntime'
['git', 'submodule', 'sync', '--recursive']
Synchronizing submodule url for 'cmake/external/DNNLibrary'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/flatbuffers'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/glog'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/onnx'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/onnx/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/onnx/third_party/pybind11'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/onnx/third_party/pybind11/tools/clang'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/protobuf'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/protobuf/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/protobuf/third_party/googletest'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/pybind11'
Synchronizing submodule url for 'cmake/external/DNNLibrary/third_party/pybind11/tools/clang'
Synchronizing submodule url for 'cmake/external/cub'
Synchronizing submodule url for 'cmake/external/date'
Synchronizing submodule url for 'cmake/external/eigen'
Synchronizing submodule url for 'cmake/external/gemmlowp'
Synchronizing submodule url for 'cmake/external/googletest'
Synchronizing submodule url for 'cmake/external/grpc'
Synchronizing submodule url for 'cmake/external/grpc/third_party/abseil-cpp'
Synchronizing submodule url for 'cmake/external/grpc/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/grpc/third_party/bloaty'
Synchronizing submodule url for 'cmake/external/grpc/third_party/bloaty/third_party/googletest'
Synchronizing submodule url for 'cmake/external/grpc/third_party/bloaty/third_party/libFuzzer'
Synchronizing submodule url for 'cmake/external/grpc/third_party/bloaty/third_party/re2'
Synchronizing submodule url for 'cmake/external/grpc/third_party/boringssl'
Synchronizing submodule url for 'cmake/external/grpc/third_party/boringssl-with-bazel'
Synchronizing submodule url for 'cmake/external/grpc/third_party/cares/cares'
Synchronizing submodule url for 'cmake/external/grpc/third_party/data-plane-api'
Synchronizing submodule url for 'cmake/external/grpc/third_party/gflags'
Synchronizing submodule url for 'cmake/external/grpc/third_party/gflags/doc'
Synchronizing submodule url for 'cmake/external/grpc/third_party/googleapis'
Synchronizing submodule url for 'cmake/external/grpc/third_party/googletest'
Synchronizing submodule url for 'cmake/external/grpc/third_party/libcxx'
Synchronizing submodule url for 'cmake/external/grpc/third_party/libcxxabi'
Synchronizing submodule url for 'cmake/external/grpc/third_party/protobuf'
Synchronizing submodule url for 'cmake/external/grpc/third_party/protobuf/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/grpc/third_party/protobuf/third_party/googletest'
Synchronizing submodule url for 'cmake/external/grpc/third_party/protoc-gen-validate'
Synchronizing submodule url for 'cmake/external/grpc/third_party/upb'
Synchronizing submodule url for 'cmake/external/grpc/third_party/upb/third_party/protobuf'
Synchronizing submodule url for 'cmake/external/grpc/third_party/upb/third_party/protobuf/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/grpc/third_party/upb/third_party/protobuf/third_party/googletest'
Synchronizing submodule url for 'cmake/external/grpc/third_party/zlib'
Synchronizing submodule url for 'cmake/external/json'
Synchronizing submodule url for 'cmake/external/mimalloc'
Synchronizing submodule url for 'cmake/external/nsync'
Synchronizing submodule url for 'cmake/external/onnx'
Synchronizing submodule url for 'cmake/external/onnx/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/onnx/third_party/pybind11'
Synchronizing submodule url for 'cmake/external/onnx/third_party/pybind11/tools/clang'
Synchronizing submodule url for 'cmake/external/onnx-tensorrt'
Synchronizing submodule url for 'cmake/external/onnx-tensorrt/third_party/onnx'
Synchronizing submodule url for 'cmake/external/onnx-tensorrt/third_party/onnx/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/onnx-tensorrt/third_party/onnx/third_party/pybind11'
Synchronizing submodule url for 'cmake/external/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang'
Synchronizing submodule url for 'cmake/external/protobuf'
Synchronizing submodule url for 'cmake/external/protobuf/third_party/benchmark'
Synchronizing submodule url for 'cmake/external/protobuf/third_party/googletest'
Synchronizing submodule url for 'cmake/external/re2'
Synchronizing submodule url for 'cmake/external/spdlog'
Synchronizing submodule url for 'cmake/external/tvm'
Synchronizing submodule url for 'cmake/external/tvm/3rdparty/HalideIR'
Synchronizing submodule url for 'cmake/external/tvm/3rdparty/dlpack'
Synchronizing submodule url for 'cmake/external/tvm/3rdparty/dmlc-core'
Synchronizing submodule url for 'cmake/external/tvm/3rdparty/rang'
Synchronizing submodule url for 'cmake/external/wil'
2020-06-08 14:41:20,636 Build [DEBUG] - Subprocess completed. Return code=0
2020-06-08 14:41:20,639 Build [DEBUG] - Running subprocess in '/home/nvidia/onnxruntime'
['git', 'submodule', 'update', '--init', '--recursive']
2020-06-08 14:41:24,414 Build [DEBUG] - Subprocess completed. Return code=0
2020-06-08 14:41:24,416 Build [INFO] - Generating CMake build tree
2020-06-08 14:41:24,417 Build [DEBUG] - Running subprocess in '/home/nvidia/onnxruntime/build/Linux/Release'
['/home/nvidia/cmake-3.13.0/bin/cmake', '/home/nvidia/onnxruntime/cmake', '-Donnxruntime_RUN_ONNX_TESTS=OFF', '-Donnxruntime_GENERATE_TEST_REPORTS=ON', '-Donnxruntime_DEV_MODE=OFF', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-Donnxruntime_USE_CUDA=ON', '-Donnxruntime_USE_NSYNC=OFF', '-Donnxruntime_CUDNN_HOME=/usr/lib/aarch64-linux-gnu', '-Donnxruntime_USE_AUTOML=OFF', '-Donnxruntime_CUDA_HOME=/usr/local/cuda', '-Donnxruntime_USE_JEMALLOC=OFF', '-Donnxruntime_USE_MIMALLOC=OFF', '-Donnxruntime_ENABLE_PYTHON=ON', '-Donnxruntime_BUILD_CSHARP=OFF', '-Donnxruntime_BUILD_JAVA=OFF', '-Donnxruntime_BUILD_SHARED_LIB=OFF', '-Donnxruntime_USE_EIGEN_FOR_BLAS=ON', '-Donnxruntime_USE_OPENBLAS=OFF', '-Donnxruntime_USE_DNNL=OFF', '-Donnxruntime_USE_MKLML=OFF', '-Donnxruntime_USE_GEMMLOWP=OFF', '-Donnxruntime_USE_NGRAPH=OFF', '-Donnxruntime_USE_OPENVINO=OFF', '-Donnxruntime_USE_OPENVINO_MYRIAD=OFF', '-Donnxruntime_USE_OPENVINO_GPU_FP32=OFF', '-Donnxruntime_USE_OPENVINO_GPU_FP16=OFF', '-Donnxruntime_USE_OPENVINO_CPU_FP32=OFF', '-Donnxruntime_USE_OPENVINO_VAD_M=OFF', '-Donnxruntime_USE_OPENVINO_VAD_F=OFF', '-Donnxruntime_USE_NNAPI=OFF', '-Donnxruntime_USE_OPENMP=ON', '-Donnxruntime_USE_TVM=OFF', '-Donnxruntime_USE_LLVM=OFF', '-Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF', '-Donnxruntime_USE_BRAINSLICE=OFF', '-Donnxruntime_USE_NUPHAR=OFF', '-Donnxruntime_USE_EIGEN_THREADPOOL=OFF', '-Donnxruntime_USE_TENSORRT=ON', '-Donnxruntime_TENSORRT_HOME=/usr/lib/aarch64-linux-gnu', '-Donnxruntime_CROSS_COMPILING=OFF', '-Donnxruntime_BUILD_SERVER=OFF', '-Donnxruntime_BUILD_x86=OFF', '-Donnxruntime_USE_FULL_PROTOBUF=ON', '-Donnxruntime_DISABLE_CONTRIB_OPS=OFF', '-Donnxruntime_MSVC_STATIC_RUNTIME=OFF', '-Donnxruntime_ENABLE_LANGUAGE_INTEROP_OPS=OFF', '-Donnxruntime_USE_DML=OFF', '-Donnxruntime_USE_TELEMETRY=OFF', '-DCUDA_CUDA_LIBRARY=/usr/local/cuda/lib64/stubs', '-Donnxruntime_PYBIND_EXPORT_OPSCHEMA=OFF', '-DCMAKE_BUILD_TYPE=Release']
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Found ZLIB: /usr/lib/aarch64-linux-gnu/libz.so (found version "1.2.11") 
-- Found PNG: /usr/lib/aarch64-linux-gnu/libpng.so (found version "1.6.34") 
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.6.9", minimum required is "3.5") 
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.6m.so (found suitable version "3.6.9", minimum required is "3.5") 
# date: USE_SYSTEM_TZ_DB ON
# date: USE_TZ_DB_IN_DOT OFF
# date: BUILD_SHARED_LIBS OFF
# date: ENABLE_DATE_TESTING OFF
-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE
-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE - Success
-- Performing Test HAS_UNUSED_PARAMETER
-- Performing Test HAS_UNUSED_PARAMETER - Success
-- Performing Test HAS_CAST_FUNCTION_TYPE
-- Performing Test HAS_CAST_FUNCTION_TYPE - Failed
-- Performing Test HAS_PARENTHESES
-- Performing Test HAS_PARENTHESES - Success
-- Performing Test HAS_USELESS_CAST
-- Performing Test HAS_USELESS_CAST - Success
-- Performing Test HAS_NONNULL_COMPARE
-- Performing Test HAS_NONNULL_COMPARE - Success
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE - Failed
-- Performing Test HAS_CATCH_VALUE
-- Performing Test HAS_CATCH_VALUE - Failed
-- Performing Test HAS_MISSING_BRACES
-- Performing Test HAS_MISSING_BRACES - Success
-- Performing Test HAS_IGNORED_ATTRIBUTES
-- Performing Test HAS_IGNORED_ATTRIBUTES - Success
-- Performing Test HAS_DEPRECATED_COPY
-- Performing Test HAS_DEPRECATED_COPY - Failed
-- Performing Test HAS_CLASS_MEMACCESS
-- Performing Test HAS_CLASS_MEMACCESS - Failed
-- The CUDA compiler identification is NVIDIA 10.2.89
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- GPU_ARCH is not defined. Generating CUDA code for default SMs.
-- Found CUDNN: /usr/include  
-- Found TensorRT headers at /usr/include/aarch64-linux-gnu
-- Find TensorRT libs at /usr/lib/aarch64-linux-gnu/libnvinfer.so;/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
-- Found TENSORRT: /usr/include/aarch64-linux-gnu  
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Found NumPy: /usr/lib/python3/dist-packages/numpy/core/include (found version "1.13.3") 
-- NumPy ver. 1.13.3 found (include: /usr/lib/python3/dist-packages/numpy/core/include)
-- Found PythonInterp: /usr/bin/python3 (found version "3.6.9") 
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CUDA_CUDA_LIBRARY
    onnxruntime_CUDA_HOME
    onnxruntime_USE_EIGEN_THREADPOOL
    onnxruntime_USE_OPENVINO_CPU_FP32
    onnxruntime_USE_OPENVINO_GPU_FP16
    onnxruntime_USE_OPENVINO_GPU_FP32
    onnxruntime_USE_OPENVINO_MYRIAD
    onnxruntime_USE_OPENVINO_VAD_F
    onnxruntime_USE_OPENVINO_VAD_M


-- Build files have been written to: /home/nvidia/onnxruntime/build/Linux/Release
2020-06-08 14:41:54,048 Build [DEBUG] - Subprocess completed. Return code=0
2020-06-08 14:41:54,050 Build [INFO] - Building targets for Release configuration
2020-06-08 14:41:54,052 Build [DEBUG] - Running subprocess in '/home/nvidia/onnxruntime'
['/home/nvidia/cmake-3.13.0/bin/cmake', '--build', '/home/nvidia/onnxruntime/build/Linux/Release', '--config', 'Release']
Scanning dependencies of target custom_op_library
[  0%] Building CXX object CMakeFiles/custom_op_library.dir/home/nvidia/onnxruntime/onnxruntime/test/testdata/custom_op_library/custom_op_library.cc.o
[  0%] Linking CXX shared library libcustom_op_library.so
[  0%] Built target custom_op_library
Scanning dependencies of target onnxruntime_mlas
[  0%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/platform.cpp.o
[  0%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/threading.cpp.o
[  0%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/dgemm.cpp.o
[  0%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/sgemm.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/qgemm.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/convolve.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/pooling.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/reorder.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/snchwc.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/activate.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/logistic.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/tanh.cpp.o
[  1%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/erf.cpp.o
[  1%] Building ASM object CMakeFiles/onnxruntime_mlas.dir/home/nvidia/onnxruntime/onnxruntime/core/mlas/lib/aarch64/SgemmKernelNeon.S.o
[  2%] Linking CXX static library libonnxruntime_mlas.a
[  2%] Built target onnxruntime_mlas
Scanning dependencies of target libprotobuf
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/arena.cc.o
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/arenastring.cc.o
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/extension_set.cc.o
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/generated_message_table_driven_lite.cc.o
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/generated_message_util.cc.o
[  2%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/implicit_weak_message.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/coded_stream.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/zero_copy_stream.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/zero_copy_stream_impl_lite.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/message_lite.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/repeated_field.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/bytestream.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/common.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/int128.cc.o
[  3%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/io_win32.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/status.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/statusor.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/stringpiece.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/stringprintf.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/structurally_valid.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/strutil.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/time.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wire_format_lite.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/any.cc.o
[  4%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/any.pb.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/api.pb.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/compiler/importer.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/compiler/parser.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/descriptor.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/descriptor.pb.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/descriptor_database.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/duration.pb.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/dynamic_message.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/empty.pb.cc.o
[  5%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/extension_set_heavy.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/field_mask.pb.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/generated_message_reflection.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/generated_message_table_driven.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/gzip_stream.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/printer.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/strtod.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/tokenizer.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/io/zero_copy_stream_impl.cc.o
[  6%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/map_field.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/message.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/reflection_ops.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/service.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/source_context.pb.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/struct.pb.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/mathlimits.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/substitute.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/text_format.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/timestamp.pb.cc.o
[  7%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/type.pb.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/unknown_field_set.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/delimited_message_util.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/field_comparator.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/field_mask_util.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/datapiece.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/default_value_objectwriter.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/error_listener.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/field_mask_utility.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_escaping.cc.o
[  8%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_objectwriter.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_stream_parser.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/object_writer.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/proto_writer.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/protostream_objectsource.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/protostream_objectwriter.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/type_info.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/type_info_test_helper.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/utility.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/json_util.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/message_differencer.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/time_util.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/type_resolver_util.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wire_format.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o
[ 11%] Linking CXX static library libprotobuf.a
[ 11%] Built target libprotobuf
Scanning dependencies of target libprotoc
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/code_generator.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/command_line_interface.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum_field.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_extension.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_field.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_file.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_generator.cc.o
[ 11%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_helpers.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_map_field.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_message.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_message_field.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_padding_optimizer.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_service.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_string_field.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_doc_comment.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_enum.cc.o
[ 12%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_enum_field.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_field_base.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_generator.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_helpers.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_map_field.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_message.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_message_field.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_primitive_field.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_reflection_class.cc.o
[ 13%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_repeated_message_field.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_repeated_primitive_field.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_source_generator_base.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/csharp/csharp_wrapper_field.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/java/java_context.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/java/java_doc_comment.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/java/java_enum.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/java/java_enum_field.cc.o
[ 14%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/java/java_enum_field_lite.cc.o

[SKIP]

[ 62%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/generator/range.cc.o
[ 62%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/gpu_data_transfer.cc.o
[ 62%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/binary_elementwise_ops.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/clip.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/gemm.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/matmul.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/softmax.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/topk.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/math/unary_elementwise_ops.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/batch_norm.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/conv.cc.o
[ 63%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/conv_transpose.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/instance_norm.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/lrn.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/pool.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/nn/shrink.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/object_detection/non_max_suppression.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/object_detection/roialign.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/reduction/reduction_ops.cc.o
[ 64%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.cc.o
In file included from /home/nvidia/onnxruntime/include/onnxruntime/core/framework/tensor.h:12:0,
                 from /home/nvidia/onnxruntime/onnxruntime/core/framework/data_transfer.h:7,
                 from /home/nvidia/onnxruntime/onnxruntime/core/framework/data_transfer_manager.h:7,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cuda_common.h:7,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cudnn_common.h:5,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h:7,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.cc:4:
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h: In member function ‘onnxruntime::common::Status onnxruntime::cuda::CudnnRNN::Set(cudnnContext* const&, int64_t, int, cudnnDropoutDescriptor_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t)’:
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h:45:27: error: ‘cudnnSetRNNDescriptor’ was not declared in this scope
     CUDNN_RETURN_IF_ERROR(cudnnSetRNNDescriptor(cudnnHandle,
                           ^
/home/nvidia/onnxruntime/include/onnxruntime/core/common/common.h:149:21: note: in definition of macro ‘ORT_RETURN_IF_ERROR_SESSIONID’
     auto _status = (expr);                   \
                     ^~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cuda_common.h:23:37: note: in expansion of macro ‘ORT_RETURN_IF_ERROR’
 #define CUDNN_RETURN_IF_ERROR(expr) ORT_RETURN_IF_ERROR(CUDNN_CALL(expr) ? common::Status::OK() : common::Status(common::ONNXRUNTIME, common::FAIL))
                                     ^~~~~~~~~~~~~~~~~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cuda_common.h:23:57: note: in expansion of macro ‘CUDNN_CALL’
 #define CUDNN_RETURN_IF_ERROR(expr) ORT_RETURN_IF_ERROR(CUDNN_CALL(expr) ? common::Status::OK() : common::Status(common::ONNXRUNTIME, common::FAIL))
                                                         ^~~~~~~~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h:45:5: note: in expansion of macro ‘CUDNN_RETURN_IF_ERROR’
     CUDNN_RETURN_IF_ERROR(cudnnSetRNNDescriptor(cudnnHandle,
     ^~~~~~~~~~~~~~~~~~~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h:45:27: note: suggested alternative: ‘cudnnSetLRNDescriptor’
     CUDNN_RETURN_IF_ERROR(cudnnSetRNNDescriptor(cudnnHandle,
                           ^
/home/nvidia/onnxruntime/include/onnxruntime/core/common/common.h:149:21: note: in definition of macro ‘ORT_RETURN_IF_ERROR_SESSIONID’
     auto _status = (expr);                   \
                     ^~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cuda_common.h:23:37: note: in expansion of macro ‘ORT_RETURN_IF_ERROR’
 #define CUDNN_RETURN_IF_ERROR(expr) ORT_RETURN_IF_ERROR(CUDNN_CALL(expr) ? common::Status::OK() : common::Status(common::ONNXRUNTIME, common::FAIL))
                                     ^~~~~~~~~~~~~~~~~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/cuda_common.h:23:57: note: in expansion of macro ‘CUDNN_CALL’
 #define CUDNN_RETURN_IF_ERROR(expr) ORT_RETURN_IF_ERROR(CUDNN_CALL(expr) ? common::Status::OK() : common::Status(common::ONNXRUNTIME, common::FAIL))
                                                         ^~~~~~~~~~
/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.h:45:5: note: in expansion of macro ‘CUDNN_RETURN_IF_ERROR’
     CUDNN_RETURN_IF_ERROR(cudnnSetRNNDescriptor(cudnnHandle,
     ^~~~~~~~~~~~~~~~~~~~~
In file included from /home/nvidia/onnxruntime/cmake/external/eigen/Eigen/Core:313:0,
                 from /home/nvidia/onnxruntime/onnxruntime/core/util/math_cpuonly.h:31,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cpu/rnn/rnn_helpers.h:22,
                 from /home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.cc:6:
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h: In member function ‘void Eigen::internal::gebp_traits<float, float, false, false, 4, 0>::updateRhs(const RhsScalar*, Eigen::internal::gebp_traits<float, float, false, false, 4, 0>::RhsPacketx4&) const’:
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1079:55: warning: unused parameter ‘b’ [-Wunused-parameter]
   EIGEN_STRONG_INLINE void updateRhs(const RhsScalar* b, RhsPacketx4& dest) const
                                                       ^
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1079:71: warning: unused parameter ‘dest’ [-Wunused-parameter]
   EIGEN_STRONG_INLINE void updateRhs(const RhsScalar* b, RhsPacketx4& dest) const
                                                                       ^~~~
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h: In member function ‘void Eigen::internal::gebp_traits<double, double, false, false, 4>::updateRhs(const RhsScalar*, Eigen::internal::gebp_traits<double, double, false, false, 4>::RhsPacketx4&) const’:
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1148:55: warning: unused parameter ‘b’ [-Wunused-parameter]
   EIGEN_STRONG_INLINE void updateRhs(const RhsScalar* b, RhsPacketx4& dest) const
                                                       ^
/home/nvidia/onnxruntime/cmake/external/eigen/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1148:71: warning: unused parameter ‘dest’ [-Wunused-parameter]
   EIGEN_STRONG_INLINE void updateRhs(const RhsScalar* b, RhsPacketx4& dest) const
                                                                       ^~~~
CMakeFiles/onnxruntime_providers_cuda.dir/build.make:465: recipe for target 'CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.cc.o' failed
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/onnxruntime/onnxruntime/core/providers/cuda/rnn/cudnn_rnn_base.cc.o] Error 1
CMakeFiles/Makefile2:952: recipe for target 'CMakeFiles/onnxruntime_providers_cuda.dir/all' failed
make[1]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
  File "/home/nvidia/onnxruntime/tools/ci_build/build.py", line 1043, in <module>
    sys.exit(main())
  File "/home/nvidia/onnxruntime/tools/ci_build/build.py", line 975, in main
    build_targets(cmake_path, build_dir, configs, args.parallel)
  File "/home/nvidia/onnxruntime/tools/ci_build/build.py", line 415, in build_targets
    run_subprocess(cmd_args)
  File "/home/nvidia/onnxruntime/tools/ci_build/build.py", line 197, in run_subprocess
    completed_process = subprocess.run(args, cwd=cwd, check=True, stdout=stdout, stderr=stderr, env=my_env, shell=shell)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/nvidia/cmake-3.13.0/bin/cmake', '--build', '/home/nvidia/onnxruntime/build/Linux/Release', '--config', 'Release']' returned non-zero exit status 2.

My cmake version is using 3.13.0.

Environment

TensorRT Version : 7.1 with Jetpack 4.4
GPU Type : (Jetson AGX Xavier)
Nvidia Driver Version : Jetpack 4.4
CUDA Version : 10.2
CUDNN Version : 8.0
Operating System + Version : Jetpack 4.4 (Costimized Ubuntu 18.04)
Python Version (if applicable) : 3.6
PyTorch Version (if applicable): 1.5.0
cmake version: 3.13.0

Relevant Files

log.txt (229.9 KB)

Steps To Reproduce

I was following from onnxruntime official instruction.

The step 4 Modify cmake/CMakeLists.txt, I still add this line

-  set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -gencode=arch=compute_70,code=sm_70")
+  set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -gencode=arch=compute_72,code=sm_72") # AGX

Is there any idea about this?
Thank you

Hi,

Thanks for reporting this to us.

This is a known issue on the cuDNN API changes in v8.0.
Please check this comment for the fix:

Thanks.

Hi @AastaLLL,

Thank you.
It is really helpful, and I fixed my problem.

BR,
Chieh