Build onnxruntime 1.19.2 fail due to API HardwareCompatibilityLevel

Accelerated Computing CUDA CUDA Setup and Installation
1

It seems onnxruntime 1.19.2 used setHardwareCompatibilityLevel,but CUDA 11.8 don’t have this API implemented.

Please tell me which version is OK to support build onnxruntime 1.19.2 on L4T 36.5 Jetpack 5.1.4?

[ 80%] Building CXX object CMakeFiles/onnx_test_runner.dir/home/daniel/Work/onnxruntime/onnxruntime/test/onnx/main.cc.o
[ 81%] Built target onnxruntime
[ 81%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/command_args_parser.cc.o
[ 81%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/main.cc.o
[ 81%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/performance_runner.cc.o
[ 81%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/ort_test_session.cc.o
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc: In member function ‘onnxruntime::common::Status onnxruntime::TensorrtExecutionProvider::CreateNodeComputeInfoFromGraph(const onnxruntime::GraphViewer&, const onnxruntime::Node&, std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int>&, std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int>&, std::vector<onnxruntime::NodeComputeInfo>&)’:
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3053:17: error: ‘class nvinfer1::IBuilderConfig’ has no member named ‘setHardwareCompatibilityLevel’
 3053 |     trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3053:57: error: ‘nvinfer1::HardwareCompatibilityLevel’ has not been declared
 3053 |     trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc: In lambda function:
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3644:21: error: ‘class nvinfer1::IBuilderConfig’ has no member named ‘setHardwareCompatibilityLevel’
 3644 |         trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3644:61: error: ‘nvinfer1::HardwareCompatibilityLevel’ has not been declared
 3644 |         trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~
[ 82%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/posix/utils.cc.o
[ 82%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/core/platform/path_lib.cc.o
make[2]: *** [CMakeFiles/onnxruntime_providers_tensorrt.dir/build.make:149: CMakeFiles/onnxruntime_providers_tensorrt.dir/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2779: CMakeFiles/onnxruntime_providers_tensorrt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 82%] Linking CXX executable onnxruntime_perf_test
[ 82%] Built target onnxruntime_perf_test
[ 82%] Linking CXX executable onnx_test_runner
[ 82%] Built target onnx_test_runner
make: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 2971, in <module>
    sys.exit(main())
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 2861, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 1729, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 860, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/daniel/Work/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/daniel/Work/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j6']' returned non-zero exit status 2.
2

We have upgrade to cuda 12.3.107, and still got setHardwareCompatibilityLevel issue.

When did CUDA implement this API?

[ 78%] Building CXX object CMakeFiles/onnxruntime_providers_tensorrt.dir/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_provider_factory.cc.o
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc: In member function ‘onnxruntime::common::Status onnxruntime::TensorrtExecutionProvider::CreateNodeComputeInfoFromGraph(const onnxruntime::GraphViewer&, const onnxruntime::Node&, std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int>&, std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int>&, std::vector<onnxruntime::NodeComputeInfo>&)’:
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3053:17: error: ‘class nvinfer1::IBuilderConfig’ has no member named ‘setHardwareCompatibilityLevel’
 3053 |     trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3053:57: error: ‘nvinfer1::HardwareCompatibilityLevel’ has not been declared
 3053 |     trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc: In lambda function:
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3644:21: error: ‘class nvinfer1::IBuilderConfig’ has no member named ‘setHardwareCompatibilityLevel’
 3644 |         trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:3644:61: error: ‘nvinfer1::HardwareCompatibilityLevel’ has not been declared
 3644 |         trt_config->setHardwareCompatibilityLevel(nvinfer1::HardwareCompatibilityLevel::kAMPERE_PLUS);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~
[ 79%] Building C object CMakeFiles/onnxruntime.dir/generated_source.c.o
[ 79%] Linking CXX shared library libonnxruntime.so
[ 79%] Linking CXX static library libabsl_flags_parse.a
[ 79%] Built target absl_flags_parse
[ 79%] Building CXX object CMakeFiles/onnx_test_runner.dir/home/daniel/Work/onnxruntime/onnxruntime/test/onnx/main.cc.o
[ 79%] Building CXX object _deps/googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 79%] Built target onnxruntime
[ 79%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/main.cc.o
[ 79%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/ort_test_session.cc.o
[ 79%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/command_args_parser.cc.o
make[2]: *** [CMakeFiles/onnxruntime_providers_tensorrt.dir/build.make:149: CMakeFiles/onnxruntime_providers_tensorrt.dir/home/daniel/Work/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2779: CMakeFiles/onnxruntime_providers_tensorrt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 79%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/performance_runner.cc.o
[ 80%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/test/perftest/posix/utils.cc.o
[ 80%] Building CXX object CMakeFiles/onnxruntime_perf_test.dir/home/daniel/Work/onnxruntime/onnxruntime/core/platform/path_lib.cc.o
[ 80%] Linking CXX executable onnx_test_runner
[ 80%] Linking CXX executable onnxruntime_perf_test
[ 80%] Built target onnxruntime_perf_test
[ 80%] Built target onnx_test_runner
[ 80%] Linking CXX static library ../../../lib/libgtest.a
[ 80%] Built target gtest
make: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 2971, in <module>
    sys.exit(main())
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 2861, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 1729, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/daniel/Work/onnxruntime/tools/ci_build/build.py", line 860, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/daniel/Work/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/daniel/Work/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j6']' returned non-zero exit status 2.