Description
I convert a model from detectron2, here is the model file region_detection_new.onnx - Google Drive. It tell me that [optimizer.cpp::computeCosts::3728] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/model/Reshape_29 + /model/Reshape_30 + /model/Transpose_10.../model/Cast_26]}.)
Environment
TensorRT Version: 8.5.3
GPU Type: GetForce GTX 3090
Nvidia Driver Version: 455.23.04
CUDA Version: 11.1
CUDNN Version: 3.4.6
Operating System + Version: ubuntu 20.04.1 LTS
Python Version (if applicable): 3.9.7
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 2.0.1
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Steps To Reproduce
trtexec --onnx=region_detection_new.onnx --saveEngine=region_detection_new.onnx
&&&& RUNNING TensorRT.trtexec [TensorRT v8503] # trtexec --onnx=region_detection_new.onnx --saveEngine=region_detection_new.engine
[11/03/2023-12:22:22] [I] === Model Options ===
[11/03/2023-12:22:22] [I] Format: ONNX
[11/03/2023-12:22:22] [I] Model: region_detection_new.onnx
[11/03/2023-12:22:22] [I] Output:
[11/03/2023-12:22:22] [I] === Build Options ===
[11/03/2023-12:22:22] [I] Max batch: explicit batch
[11/03/2023-12:22:22] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[11/03/2023-12:22:22] [I] minTiming: 1
[11/03/2023-12:22:22] [I] avgTiming: 8
[11/03/2023-12:22:22] [I] Precision: FP32
[11/03/2023-12:22:22] [I] LayerPrecisions:
[11/03/2023-12:22:22] [I] Calibration:
[11/03/2023-12:22:22] [I] Refit: Disabled
[11/03/2023-12:22:22] [I] Sparsity: Disabled
[11/03/2023-12:22:22] [I] Safe mode: Disabled
[11/03/2023-12:22:22] [I] DirectIO mode: Disabled
[11/03/2023-12:22:22] [I] Restricted mode: Disabled
[11/03/2023-12:22:22] [I] Build only: Disabled
[11/03/2023-12:22:22] [I] Save engine: region_detection_new.engine
[11/03/2023-12:22:22] [I] Load engine:
[11/03/2023-12:22:22] [I] Profiling verbosity: 0
[11/03/2023-12:22:22] [I] Tactic sources: Using default tactic sources
[11/03/2023-12:22:22] [I] timingCacheMode: local
[11/03/2023-12:22:22] [I] timingCacheFile:
[11/03/2023-12:22:22] [I] Heuristic: Disabled
[11/03/2023-12:22:22] [I] Preview Features: Use default preview flags.
[11/03/2023-12:22:22] [I] Input(s)s format: fp32:CHW
[11/03/2023-12:22:22] [I] Output(s)s format: fp32:CHW
[11/03/2023-12:22:22] [I] Input build shapes: model
[11/03/2023-12:22:22] [I] Input calibration shapes: model
[11/03/2023-12:22:22] [I] === System Options ===
[11/03/2023-12:22:22] [I] Device: 0
[11/03/2023-12:22:22] [I] DLACore:
[11/03/2023-12:22:22] [I] Plugins:
[11/03/2023-12:22:22] [I] === Inference Options ===
[11/03/2023-12:22:22] [I] Batch: Explicit
[11/03/2023-12:22:22] [I] Input inference shapes: model
[11/03/2023-12:22:22] [I] Iterations: 10
[11/03/2023-12:22:22] [I] Duration: 3s (+ 200ms warm up)
[11/03/2023-12:22:22] [I] Sleep time: 0ms
[11/03/2023-12:22:22] [I] Idle time: 0ms
[11/03/2023-12:22:22] [I] Streams: 1
[11/03/2023-12:22:22] [I] ExposeDMA: Disabled
[11/03/2023-12:22:22] [I] Data transfers: Enabled
[11/03/2023-12:22:22] [I] Spin-wait: Disabled
[11/03/2023-12:22:22] [I] Multithreading: Disabled
[11/03/2023-12:22:22] [I] CUDA Graph: Disabled
[11/03/2023-12:22:22] [I] Separate profiling: Disabled
[11/03/2023-12:22:22] [I] Time Deserialize: Disabled
[11/03/2023-12:22:22] [I] Time Refit: Disabled
[11/03/2023-12:22:22] [I] NVTX verbosity: 0
[11/03/2023-12:22:22] [I] Persistent Cache Ratio: 0
[11/03/2023-12:22:22] [I] Inputs:
[11/03/2023-12:22:22] [I] === Reporting Options ===
[11/03/2023-12:22:22] [I] Verbose: Disabled
[11/03/2023-12:22:22] [I] Averages: 10 inferences
[11/03/2023-12:22:22] [I] Percentiles: 90,95,99
[11/03/2023-12:22:22] [I] Dump refittable layers:Disabled
[11/03/2023-12:22:22] [I] Dump output: Disabled
[11/03/2023-12:22:22] [I] Profile: Disabled
[11/03/2023-12:22:22] [I] Export timing to JSON file:
[11/03/2023-12:22:22] [I] Export output to JSON file:
[11/03/2023-12:22:22] [I] Export profile to JSON file:
[11/03/2023-12:22:22] [I]
[11/03/2023-12:22:22] [I] === Device Information ===
[11/03/2023-12:22:22] [I] Selected Device: GeForce RTX 3090
[11/03/2023-12:22:22] [I] Compute Capability: 8.6
[11/03/2023-12:22:22] [I] SMs: 82
[11/03/2023-12:22:22] [I] Compute Clock Rate: 1.695 GHz
[11/03/2023-12:22:22] [I] Device Global Memory: 24268 MiB
[11/03/2023-12:22:22] [I] Shared Memory per SM: 100 KiB
[11/03/2023-12:22:22] [I] Memory Bus Width: 384 bits (ECC disabled)
[11/03/2023-12:22:22] [I] Memory Clock Rate: 9.751 GHz
[11/03/2023-12:22:22] [I]
[11/03/2023-12:22:22] [I] TensorRT version: 8.5.3
[11/03/2023-12:22:23] [I] [TRT] [MemUsageChange] Init CUDA: CPU +328, GPU +0, now: CPU 339, GPU 8578 (MiB)
[11/03/2023-12:22:27] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +445, GPU +118, now: CPU 838, GPU 11244 (MiB)
[11/03/2023-12:22:27] [W] [TRT] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
[11/03/2023-12:22:27] [I] Start parsing network model
[11/03/2023-12:22:27] [I] [TRT] ----------------------------------------------------------------
[11/03/2023-12:22:27] [I] [TRT] Input filename: region_detection_new.onnx
[11/03/2023-12:22:27] [I] [TRT] ONNX IR version: 0.0.8
[11/03/2023-12:22:27] [I] [TRT] Opset version: 14
[11/03/2023-12:22:27] [I] [TRT] Producer name: pytorch
[11/03/2023-12:22:27] [I] [TRT] Producer version: 2.0.1
[11/03/2023-12:22:27] [I] [TRT] Domain:
[11/03/2023-12:22:27] [I] [TRT] Model version: 0
[11/03/2023-12:22:27] [I] [TRT] Doc string:
[11/03/2023-12:22:27] [I] [TRT] ----------------------------------------------------------------
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:377: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:27] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:28] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[11/03/2023-12:22:36] [I] Finish parsing network model
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:36] [W] [TRT] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
[11/03/2023-12:22:41] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +625, GPU +260, now: CPU 1517, GPU 11436 (MiB)
[11/03/2023-12:22:42] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +127, GPU +58, now: CPU 1644, GPU 11494 (MiB)
[11/03/2023-12:22:41] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +625, GPU +260, now: CPU 1517, GPU 11436 (MiB)
[11/03/2023-12:22:42] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +127, GPU +58, now: CPU 1644, GPU 11494 (MiB)
[11/03/2023-12:22:42] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/03/2023-12:23:21] [W] [TRT] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_9[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_7[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_1[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_3[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_6[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_5[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_4[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero_2[size]))
[11/03/2023-12:23:21] [W] [TRT] (# 0 (VALUE /model/NonZero[size]))
[11/03/2023-12:23:46] [W] [TRT] Skipping tactic 0x0000000000000000 due to exception dimension on axis 0 of shape [8,1] must be greater than or equal to the attribute k=16
[11/03/2023-12:23:46] [E] Error[10]: [optimizer.cpp::computeCosts::3728] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/model/Reshape_29 + /model/Reshape_30 + /model/Transpose_10.../model/Cast_26]}.)
[11/03/2023-12:23:46] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[11/03/2023-12:23:46] [E] Engine could not be created from network
[11/03/2023-12:23:46] [E] Building engine failed
[11/03/2023-12:23:46] [E] Failed to create engine from model or file.
[11/03/2023-12:23:46] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8503] # trtexec --onnx=region_detection_new.onnx --saveEngine=region_detection_new.engine