Description
Hi, I tried to convert onnx to trt on JETSON ORIN NX(jetpack 5.1.1, trt 8.2.1.8, cuda 11.4.315), but got an internal error.
/usr/src/tensorrt/bin/trtexec --onnx=yolov8s.onnx --saveEngine=yolov8s.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=yolov8s.onnx --saveEngine=yolov8s.engine
[08/21/2024-11:02:43] [I] === Model Options ===
[08/21/2024-11:02:43] [I] Format: ONNX
[08/21/2024-11:02:43] [I] Model: yolov8s.onnx
[08/21/2024-11:02:43] [I] Output:
[08/21/2024-11:02:43] [I] === Build Options ===
[08/21/2024-11:02:43] [I] Max batch: explicit batch
[08/21/2024-11:02:43] [I] Workspace: 16 MiB
[08/21/2024-11:02:43] [I] minTiming: 1
[08/21/2024-11:02:43] [I] avgTiming: 8
[08/21/2024-11:02:43] [I] Precision: FP32
[08/21/2024-11:02:43] [I] Calibration:
[08/21/2024-11:02:43] [I] Refit: Disabled
[08/21/2024-11:02:43] [I] Sparsity: Disabled
[08/21/2024-11:02:43] [I] Safe mode: Disabled
[08/21/2024-11:02:43] [I] DirectIO mode: Disabled
[08/21/2024-11:02:43] [I] Restricted mode: Disabled
[08/21/2024-11:02:43] [I] Save engine: yolov8s.engine
[08/21/2024-11:02:43] [I] Load engine:
[08/21/2024-11:02:43] [I] Profiling verbosity: 0
[08/21/2024-11:02:43] [I] Tactic sources: Using default tactic sources
[08/21/2024-11:02:43] [I] timingCacheMode: local
[08/21/2024-11:02:43] [I] timingCacheFile:
[08/21/2024-11:02:43] [I] Input(s)s format: fp32:CHW
[08/21/2024-11:02:43] [I] Output(s)s format: fp32:CHW
[08/21/2024-11:02:43] [I] Input build shapes: model
[08/21/2024-11:02:43] [I] Input calibration shapes: model
[08/21/2024-11:02:43] [I] === System Options ===
[08/21/2024-11:02:43] [I] Device: 0
[08/21/2024-11:02:43] [I] DLACore:
[08/21/2024-11:02:43] [I] Plugins:
[08/21/2024-11:02:43] [I] === Inference Options ===
[08/21/2024-11:02:43] [I] Batch: Explicit
[08/21/2024-11:02:43] [I] Input inference shapes: model
[08/21/2024-11:02:43] [I] Iterations: 10
[08/21/2024-11:02:43] [I] Duration: 3s (+ 200ms warm up)
[08/21/2024-11:02:43] [I] Sleep time: 0ms
[08/21/2024-11:02:43] [I] Idle time: 0ms
[08/21/2024-11:02:43] [I] Streams: 1
[08/21/2024-11:02:43] [I] ExposeDMA: Disabled
[08/21/2024-11:02:43] [I] Data transfers: Enabled
[08/21/2024-11:02:43] [I] Spin-wait: Disabled
[08/21/2024-11:02:43] [I] Multithreading: Disabled
[08/21/2024-11:02:43] [I] CUDA Graph: Disabled
[08/21/2024-11:02:43] [I] Separate profiling: Disabled
[08/21/2024-11:02:43] [I] Time Deserialize: Disabled
[08/21/2024-11:02:43] [I] Time Refit: Disabled
[08/21/2024-11:02:43] [I] Skip inference: Disabled
[08/21/2024-11:02:43] [I] Inputs:
[08/21/2024-11:02:43] [I] === Reporting Options ===
[08/21/2024-11:02:43] [I] Verbose: Disabled
[08/21/2024-11:02:43] [I] Averages: 10 inferences
[08/21/2024-11:02:43] [I] Percentile: 99
[08/21/2024-11:02:43] [I] Dump refittable layers:Disabled
[08/21/2024-11:02:43] [I] Dump output: Disabled
[08/21/2024-11:02:43] [I] Profile: Disabled
[08/21/2024-11:02:43] [I] Export timing to JSON file:
[08/21/2024-11:02:43] [I] Export output to JSON file:
[08/21/2024-11:02:43] [I] Export profile to JSON file:
[08/21/2024-11:02:43] [I]
[08/21/2024-11:02:43] [I] === Device Information ===
[08/21/2024-11:02:43] [I] Selected Device: Orin
[08/21/2024-11:02:43] [I] Compute Capability: 8.7
[08/21/2024-11:02:43] [I] SMs: 8
[08/21/2024-11:02:43] [I] Compute Clock Rate: 0.765 GHz
[08/21/2024-11:02:43] [I] Device Global Memory: 7336 MiB
[08/21/2024-11:02:43] [I] Shared Memory per SM: 164 KiB
[08/21/2024-11:02:43] [I] Memory Bus Width: 64 bits (ECC disabled)
[08/21/2024-11:02:43] [I] Memory Clock Rate: 0.765 GHz
[08/21/2024-11:02:43] [I]
[08/21/2024-11:02:43] [I] TensorRT version: 8.2.1
[08/21/2024-11:03:41] [I] [TRT] [MemUsageChange] Init CUDA: CPU +141, GPU +0, now: CPU 152, GPU 3503 (MiB)
[08/21/2024-11:03:42] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 152 MiB, GPU 3490 MiB
[08/21/2024-11:03:42] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 224 MiB, GPU 3489 MiB
[08/21/2024-11:03:42] [I] Start parsing network model
[08/21/2024-11:03:42] [I] [TRT] ----------------------------------------------------------------
[08/21/2024-11:03:42] [I] [TRT] Input filename: yolov8s.onnx
[08/21/2024-11:03:42] [I] [TRT] ONNX IR version: 0.0.6
[08/21/2024-11:03:42] [I] [TRT] Opset version: 11
[08/21/2024-11:03:42] [I] [TRT] Producer name: pytorch
[08/21/2024-11:03:42] [I] [TRT] Producer version: 2.2.2
[08/21/2024-11:03:42] [I] [TRT] Domain:
[08/21/2024-11:03:42] [I] [TRT] Model version: 0
[08/21/2024-11:03:42] [I] [TRT] Doc string:
[08/21/2024-11:03:42] [I] [TRT] ----------------------------------------------------------------
[08/21/2024-11:03:42] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/21/2024-11:03:42] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[08/21/2024-11:03:42] [I] [TRT] /model.22/MatMul: broadcasting input1 to make tensors conform, dims(input0)=[1,4,8400,16][NONE] dims(input1)=[1,1,16,1][NONE].
[08/21/2024-11:03:42] [I] [TRT] No importer registered for op: EfficientNMS_TRT. Attempting to import as plugin.
[08/21/2024-11:03:42] [I] [TRT] Searching for plugin: EfficientNMS_TRT, plugin_version: 1, plugin_namespace:
[08/21/2024-11:03:42] [I] [TRT] Successfully created plugin: EfficientNMS_TRT
[08/21/2024-11:03:42] [I] Finish parsing network model
[08/21/2024-11:03:42] [I] [TRT] /model.22/MatMul: broadcasting input1 to make tensors conform, dims(input0)=[1,4,8400,16][NONE] dims(input1)=[1,1,16,1][NONE].
[08/21/2024-11:03:44] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +534, GPU +610, now: CPU 808, GPU 4087 (MiB)
[08/21/2024-11:03:44] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +83, GPU +132, now: CPU 891, GPU 4219 (MiB)
[08/21/2024-11:03:44] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[08/21/2024-11:03:44] [E] Error[2]: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{2055, 8, 8}, 5869},)
[08/21/2024-11:03:44] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[08/21/2024-11:03:44] [E] Engine could not be created from network
[08/21/2024-11:03:44] [E] Building engine failed
[08/21/2024-11:03:44] [E] Failed to create engine from model.
[08/21/2024-11:03:44] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=yolov8s.onnx --saveEngine=yolov8s.engine
Environment
TensorRT Version: 8.2.1.8
GPU Type: NVIDIA jetson orin NX(jetpack 5.1.1)
CUDA Version: 11.4.315
CUDNN Version: 1.0
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):