Trying to convert Yolov8.onnx into trt ( TensorRT version : 8.2, jetson-jetpack : 4.6.1)

uditbajpai0495 · April 21, 2023, 6:18am

I’m using trtexec command:
‘trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=input:1x3x640x640 --optShapes=input:1x3x640x640 --maxShapes=input:1x3x640x640 --buildOnly --saveEngine=vids.engine’

output on terminal after executing command—

&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=input:1x3x640x640 --optShapes=input:1x3x640x640 --maxShapes=input:1x3x640x640 --buildOnly --saveEngine=vids.engine
[04/21/2023-11:44:04] [I] === Model Options ===
[04/21/2023-11:44:04] [I] Format: ONNX
[04/21/2023-11:44:04] [I] Model: best.onnx
[04/21/2023-11:44:04] [I] Output:
[04/21/2023-11:44:04] [I] === Build Options ===
[04/21/2023-11:44:04] [I] Max batch: explicit batch
[04/21/2023-11:44:04] [I] Workspace: 64 MiB
[04/21/2023-11:44:04] [I] minTiming: 1
[04/21/2023-11:44:04] [I] avgTiming: 8
[04/21/2023-11:44:04] [I] Precision: FP32+FP16
[04/21/2023-11:44:04] [I] Calibration:
[04/21/2023-11:44:04] [I] Refit: Disabled
[04/21/2023-11:44:04] [I] Sparsity: Disabled
[04/21/2023-11:44:04] [I] Safe mode: Disabled
[04/21/2023-11:44:04] [I] DirectIO mode: Disabled
[04/21/2023-11:44:04] [I] Restricted mode: Disabled
[04/21/2023-11:44:04] [I] Save engine: vids.engine
[04/21/2023-11:44:04] [I] Load engine:
[04/21/2023-11:44:04] [I] Profiling verbosity: 0
[04/21/2023-11:44:04] [I] Tactic sources: Using default tactic sources
[04/21/2023-11:44:04] [I] timingCacheMode: local
[04/21/2023-11:44:04] [I] timingCacheFile:
[04/21/2023-11:44:04] [I] Input(s)s format: fp32:CHW
[04/21/2023-11:44:04] [I] Output(s)s format: fp32:CHW
[04/21/2023-11:44:04] [I] Input build shape: input=1x3x640x640+1x3x640x640+1x3x640x640
[04/21/2023-11:44:04] [I] Input calibration shapes: model
[04/21/2023-11:44:04] [I] === System Options ===
[04/21/2023-11:44:04] [I] Device: 0
[04/21/2023-11:44:04] [I] DLACore:
[04/21/2023-11:44:04] [I] Plugins:
[04/21/2023-11:44:04] [I] === Inference Options ===
[04/21/2023-11:44:04] [I] Batch: Explicit
[04/21/2023-11:44:04] [I] Input inference shape: input=1x3x640x640
[04/21/2023-11:44:04] [I] Iterations: 10
[04/21/2023-11:44:04] [I] Duration: 3s (+ 200ms warm up)
[04/21/2023-11:44:04] [I] Sleep time: 0ms
[04/21/2023-11:44:04] [I] Idle time: 0ms
[04/21/2023-11:44:04] [I] Streams: 1
[04/21/2023-11:44:04] [I] ExposeDMA: Disabled
[04/21/2023-11:44:04] [I] Data transfers: Enabled
[04/21/2023-11:44:04] [I] Spin-wait: Disabled
[04/21/2023-11:44:04] [I] Multithreading: Disabled
[04/21/2023-11:44:04] [I] CUDA Graph: Disabled
[04/21/2023-11:44:04] [I] Separate profiling: Disabled
[04/21/2023-11:44:04] [I] Time Deserialize: Disabled
[04/21/2023-11:44:04] [I] Time Refit: Disabled
[04/21/2023-11:44:04] [I] Skip inference: Enabled
[04/21/2023-11:44:04] [I] Inputs:
[04/21/2023-11:44:04] [I] === Reporting Options ===
[04/21/2023-11:44:04] [I] Verbose: Disabled
[04/21/2023-11:44:04] [I] Averages: 10 inferences
[04/21/2023-11:44:04] [I] Percentile: 99
[04/21/2023-11:44:04] [I] Dump refittable layers:Disabled
[04/21/2023-11:44:04] [I] Dump output: Disabled
[04/21/2023-11:44:04] [I] Profile: Disabled
[04/21/2023-11:44:04] [I] Export timing to JSON file:
[04/21/2023-11:44:04] [I] Export output to JSON file:
[04/21/2023-11:44:04] [I] Export profile to JSON file:
[04/21/2023-11:44:04] [I]
[04/21/2023-11:44:04] [I] === Device Information ===
[04/21/2023-11:44:04] [I] Selected Device: Xavier
[04/21/2023-11:44:04] [I] Compute Capability: 7.2
[04/21/2023-11:44:04] [I] SMs: 6
[04/21/2023-11:44:04] [I] Compute Clock Rate: 1.109 GHz
[04/21/2023-11:44:04] [I] Device Global Memory: 15825 MiB
[04/21/2023-11:44:04] [I] Shared Memory per SM: 96 KiB
[04/21/2023-11:44:04] [I] Memory Bus Width: 256 bits (ECC disabled)
[04/21/2023-11:44:04] [I] Memory Clock Rate: 1.109 GHz
[04/21/2023-11:44:04] [I]
[04/21/2023-11:44:04] [I] TensorRT version: 8.2.1
[04/21/2023-11:44:06] [I] [TRT] [MemUsageChange] Init CUDA: CPU +362, GPU +0, now: CPU 381, GPU 2950 (MiB)
[04/21/2023-11:44:07] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 381 MiB, GPU 2950 MiB
[04/21/2023-11:44:07] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 486 MiB, GPU 3058 MiB
[04/21/2023-11:44:07] [I] Start parsing network model
[04/21/2023-11:44:07] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-11:44:07] [I] [TRT] Input filename: best.onnx
[04/21/2023-11:44:07] [I] [TRT] ONNX IR version: 0.0.6
[04/21/2023-11:44:07] [I] [TRT] Opset version: 11
[04/21/2023-11:44:07] [I] [TRT] Producer name: pytorch
[04/21/2023-11:44:07] [I] [TRT] Producer version: 2.0.0
[04/21/2023-11:44:07] [I] [TRT] Domain:
[04/21/2023-11:44:07] [I] [TRT] Model version: 0
[04/21/2023-11:44:07] [I] [TRT] Doc string:
[04/21/2023-11:44:07] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-11:44:08] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[04/21/2023-11:44:08] [I] Finish parsing network model
[04/21/2023-11:44:08] [E] Cannot find input tensor with name “input” in the network inputs! Please make sure the input tensor names are correct.
[04/21/2023-11:44:08] [E] Network And Config setup failed
[04/21/2023-11:44:08] [E] Building engine failed
[04/21/2023-11:44:08] [E] Failed to create engine from model.
[04/21/2023-11:44:08] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=input:1x3x640x640 --optShapes=input:1x3x640x640 --maxShapes=input:1x3x640x640 --buildOnly --saveEngine=vids.engine

AastaLLL · April 21, 2023, 7:17am

Hi,

The input layer name is incorrect in your command.
You can find the name by visualizing the model with the below link:
https://netron.app/

Thanks.

uditbajpai0495 · April 21, 2023, 7:42am

now getting this:

trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine
[04/21/2023-13:11:47] [I] === Model Options ===
[04/21/2023-13:11:47] [I] Format: ONNX
[04/21/2023-13:11:47] [I] Model: best.onnx
[04/21/2023-13:11:47] [I] Output:
[04/21/2023-13:11:47] [I] === Build Options ===
[04/21/2023-13:11:47] [I] Max batch: explicit batch
[04/21/2023-13:11:47] [I] Workspace: 64 MiB
[04/21/2023-13:11:47] [I] minTiming: 1
[04/21/2023-13:11:47] [I] avgTiming: 8
[04/21/2023-13:11:47] [I] Precision: FP32+FP16
[04/21/2023-13:11:47] [I] Calibration:
[04/21/2023-13:11:47] [I] Refit: Disabled
[04/21/2023-13:11:47] [I] Sparsity: Disabled
[04/21/2023-13:11:47] [I] Safe mode: Disabled
[04/21/2023-13:11:47] [I] DirectIO mode: Disabled
[04/21/2023-13:11:47] [I] Restricted mode: Disabled
[04/21/2023-13:11:47] [I] Save engine: vids.engine
[04/21/2023-13:11:47] [I] Load engine:
[04/21/2023-13:11:47] [I] Profiling verbosity: 0
[04/21/2023-13:11:47] [I] Tactic sources: Using default tactic sources
[04/21/2023-13:11:47] [I] timingCacheMode: local
[04/21/2023-13:11:47] [I] timingCacheFile:
[04/21/2023-13:11:47] [I] Input(s)s format: fp32:CHW
[04/21/2023-13:11:47] [I] Output(s)s format: fp32:CHW
[04/21/2023-13:11:47] [I] Input build shape: images=1x3x640x640+1x3x640x640+1x3x640x640
[04/21/2023-13:11:47] [I] Input calibration shapes: model
[04/21/2023-13:11:47] [I] === System Options ===
[04/21/2023-13:11:47] [I] Device: 0
[04/21/2023-13:11:47] [I] DLACore:
[04/21/2023-13:11:47] [I] Plugins:
[04/21/2023-13:11:47] [I] === Inference Options ===
[04/21/2023-13:11:47] [I] Batch: Explicit
[04/21/2023-13:11:47] [I] Input inference shape: images=1x3x640x640
[04/21/2023-13:11:47] [I] Iterations: 10
[04/21/2023-13:11:47] [I] Duration: 3s (+ 200ms warm up)
[04/21/2023-13:11:47] [I] Sleep time: 0ms
[04/21/2023-13:11:47] [I] Idle time: 0ms
[04/21/2023-13:11:47] [I] Streams: 1
[04/21/2023-13:11:47] [I] ExposeDMA: Disabled
[04/21/2023-13:11:47] [I] Data transfers: Enabled
[04/21/2023-13:11:47] [I] Spin-wait: Disabled
[04/21/2023-13:11:47] [I] Multithreading: Disabled
[04/21/2023-13:11:47] [I] CUDA Graph: Disabled
[04/21/2023-13:11:47] [I] Separate profiling: Disabled
[04/21/2023-13:11:47] [I] Time Deserialize: Disabled
[04/21/2023-13:11:47] [I] Time Refit: Disabled
[04/21/2023-13:11:47] [I] Skip inference: Enabled
[04/21/2023-13:11:47] [I] Inputs:
[04/21/2023-13:11:47] [I] === Reporting Options ===
[04/21/2023-13:11:47] [I] Verbose: Disabled
[04/21/2023-13:11:47] [I] Averages: 10 inferences
[04/21/2023-13:11:47] [I] Percentile: 99
[04/21/2023-13:11:47] [I] Dump refittable layers:Disabled
[04/21/2023-13:11:47] [I] Dump output: Disabled
[04/21/2023-13:11:47] [I] Profile: Disabled
[04/21/2023-13:11:47] [I] Export timing to JSON file:
[04/21/2023-13:11:47] [I] Export output to JSON file:
[04/21/2023-13:11:47] [I] Export profile to JSON file:
[04/21/2023-13:11:47] [I]
[04/21/2023-13:11:47] [I] === Device Information ===
[04/21/2023-13:11:47] [I] Selected Device: Xavier
[04/21/2023-13:11:47] [I] Compute Capability: 7.2
[04/21/2023-13:11:47] [I] SMs: 6
[04/21/2023-13:11:47] [I] Compute Clock Rate: 1.109 GHz
[04/21/2023-13:11:47] [I] Device Global Memory: 15825 MiB
[04/21/2023-13:11:47] [I] Shared Memory per SM: 96 KiB
[04/21/2023-13:11:47] [I] Memory Bus Width: 256 bits (ECC disabled)
[04/21/2023-13:11:47] [I] Memory Clock Rate: 1.109 GHz
[04/21/2023-13:11:47] [I]
[04/21/2023-13:11:47] [I] TensorRT version: 8.2.1
[04/21/2023-13:11:49] [I] [TRT] [MemUsageChange] Init CUDA: CPU +362, GPU +0, now: CPU 381, GPU 3391 (MiB)
[04/21/2023-13:11:49] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 381 MiB, GPU 3391 MiB
[04/21/2023-13:11:50] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 486 MiB, GPU 3496 MiB
[04/21/2023-13:11:50] [I] Start parsing network model
[04/21/2023-13:11:50] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-13:11:50] [I] [TRT] Input filename: best.onnx
[04/21/2023-13:11:50] [I] [TRT] ONNX IR version: 0.0.6
[04/21/2023-13:11:50] [I] [TRT] Opset version: 11
[04/21/2023-13:11:50] [I] [TRT] Producer name: pytorch
[04/21/2023-13:11:50] [I] [TRT] Producer version: 2.0.0
[04/21/2023-13:11:50] [I] [TRT] Domain:
[04/21/2023-13:11:50] [I] [TRT] Model version: 0
[04/21/2023-13:11:50] [I] [TRT] Doc string:
[04/21/2023-13:11:50] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-13:11:50] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[04/21/2023-13:11:50] [I] Finish parsing network model
[04/21/2023-13:11:50] [E] Static model does not take explicit shapes since the shape of inference tensors will be determined by the model itself
[04/21/2023-13:11:50] [E] Network And Config setup failed
[04/21/2023-13:11:50] [E] Building engine failed
[04/21/2023-13:11:50] [E] Failed to create engine from model.
[04/21/2023-13:11:50] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine

uditbajpai0495 · April 21, 2023, 8:35am

after change the input name according to onnx model, getting this :

trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine
[04/21/2023-13:11:47] [I] === Model Options ===
[04/21/2023-13:11:47] [I] Format: ONNX
[04/21/2023-13:11:47] [I] Model: best.onnx
[04/21/2023-13:11:47] [I] Output:
[04/21/2023-13:11:47] [I] === Build Options ===
[04/21/2023-13:11:47] [I] Max batch: explicit batch
[04/21/2023-13:11:47] [I] Workspace: 64 MiB
[04/21/2023-13:11:47] [I] minTiming: 1
[04/21/2023-13:11:47] [I] avgTiming: 8
[04/21/2023-13:11:47] [I] Precision: FP32+FP16
[04/21/2023-13:11:47] [I] Calibration:
[04/21/2023-13:11:47] [I] Refit: Disabled
[04/21/2023-13:11:47] [I] Sparsity: Disabled
[04/21/2023-13:11:47] [I] Safe mode: Disabled
[04/21/2023-13:11:47] [I] DirectIO mode: Disabled
[04/21/2023-13:11:47] [I] Restricted mode: Disabled
[04/21/2023-13:11:47] [I] Save engine: vids.engine
[04/21/2023-13:11:47] [I] Load engine:
[04/21/2023-13:11:47] [I] Profiling verbosity: 0
[04/21/2023-13:11:47] [I] Tactic sources: Using default tactic sources
[04/21/2023-13:11:47] [I] timingCacheMode: local
[04/21/2023-13:11:47] [I] timingCacheFile:
[04/21/2023-13:11:47] [I] Input(s)s format: fp32:CHW
[04/21/2023-13:11:47] [I] Output(s)s format: fp32:CHW
[04/21/2023-13:11:47] [I] Input build shape: images=1x3x640x640+1x3x640x640+1x3x640x640
[04/21/2023-13:11:47] [I] Input calibration shapes: model
[04/21/2023-13:11:47] [I] === System Options ===
[04/21/2023-13:11:47] [I] Device: 0
[04/21/2023-13:11:47] [I] DLACore:
[04/21/2023-13:11:47] [I] Plugins:
[04/21/2023-13:11:47] [I] === Inference Options ===
[04/21/2023-13:11:47] [I] Batch: Explicit
[04/21/2023-13:11:47] [I] Input inference shape: images=1x3x640x640
[04/21/2023-13:11:47] [I] Iterations: 10
[04/21/2023-13:11:47] [I] Duration: 3s (+ 200ms warm up)
[04/21/2023-13:11:47] [I] Sleep time: 0ms
[04/21/2023-13:11:47] [I] Idle time: 0ms
[04/21/2023-13:11:47] [I] Streams: 1
[04/21/2023-13:11:47] [I] ExposeDMA: Disabled
[04/21/2023-13:11:47] [I] Data transfers: Enabled
[04/21/2023-13:11:47] [I] Spin-wait: Disabled
[04/21/2023-13:11:47] [I] Multithreading: Disabled
[04/21/2023-13:11:47] [I] CUDA Graph: Disabled
[04/21/2023-13:11:47] [I] Separate profiling: Disabled
[04/21/2023-13:11:47] [I] Time Deserialize: Disabled
[04/21/2023-13:11:47] [I] Time Refit: Disabled
[04/21/2023-13:11:47] [I] Skip inference: Enabled
[04/21/2023-13:11:47] [I] Inputs:
[04/21/2023-13:11:47] [I] === Reporting Options ===
[04/21/2023-13:11:47] [I] Verbose: Disabled
[04/21/2023-13:11:47] [I] Averages: 10 inferences
[04/21/2023-13:11:47] [I] Percentile: 99
[04/21/2023-13:11:47] [I] Dump refittable layers:Disabled
[04/21/2023-13:11:47] [I] Dump output: Disabled
[04/21/2023-13:11:47] [I] Profile: Disabled
[04/21/2023-13:11:47] [I] Export timing to JSON file:
[04/21/2023-13:11:47] [I] Export output to JSON file:
[04/21/2023-13:11:47] [I] Export profile to JSON file:
[04/21/2023-13:11:47] [I]
[04/21/2023-13:11:47] [I] === Device Information ===
[04/21/2023-13:11:47] [I] Selected Device: Xavier
[04/21/2023-13:11:47] [I] Compute Capability: 7.2
[04/21/2023-13:11:47] [I] SMs: 6
[04/21/2023-13:11:47] [I] Compute Clock Rate: 1.109 GHz
[04/21/2023-13:11:47] [I] Device Global Memory: 15825 MiB
[04/21/2023-13:11:47] [I] Shared Memory per SM: 96 KiB
[04/21/2023-13:11:47] [I] Memory Bus Width: 256 bits (ECC disabled)
[04/21/2023-13:11:47] [I] Memory Clock Rate: 1.109 GHz
[04/21/2023-13:11:47] [I]
[04/21/2023-13:11:47] [I] TensorRT version: 8.2.1
[04/21/2023-13:11:49] [I] [TRT] [MemUsageChange] Init CUDA: CPU +362, GPU +0, now: CPU 381, GPU 3391 (MiB)
[04/21/2023-13:11:49] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 381 MiB, GPU 3391 MiB
[04/21/2023-13:11:50] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 486 MiB, GPU 3496 MiB
[04/21/2023-13:11:50] [I] Start parsing network model
[04/21/2023-13:11:50] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-13:11:50] [I] [TRT] Input filename: best.onnx
[04/21/2023-13:11:50] [I] [TRT] ONNX IR version: 0.0.6
[04/21/2023-13:11:50] [I] [TRT] Opset version: 11
[04/21/2023-13:11:50] [I] [TRT] Producer name: pytorch
[04/21/2023-13:11:50] [I] [TRT] Producer version: 2.0.0
[04/21/2023-13:11:50] [I] [TRT] Domain:
[04/21/2023-13:11:50] [I] [TRT] Model version: 0
[04/21/2023-13:11:50] [I] [TRT] Doc string:
[04/21/2023-13:11:50] [I] [TRT] ----------------------------------------------------------------
[04/21/2023-13:11:50] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[04/21/2023-13:11:50] [I] Finish parsing network model
[04/21/2023-13:11:50] [E] Static model does not take explicit shapes since the shape of inference tensors will be determined by the model itself
[04/21/2023-13:11:50] [E] Network And Config setup failed
[04/21/2023-13:11:50] [E] Building engine failed
[04/21/2023-13:11:50] [E] Failed to create engine from model.
[04/21/2023-13:11:50] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --workspace=64 --minShapes=images:1x3x640x640 --optShapes=images:1x3x640x640 --maxShapes=images:1x3x640x640 --buildOnly --saveEngine=vids.engine

uditbajpai0495 · April 24, 2023, 4:36am

tell me how can I solve this problem

AastaLLL · April 24, 2023, 6:27am

Hi,

Please try the following command:

$ trtexec --onnx=best.onnx --fp16 --buildOnly --saveEngine=vids.engine

Thanks.

uditbajpai0495 · April 24, 2023, 7:09am

now getting this :

trtexec --onnx=best.onnx --fp16 --buildOnly --saveEngine=yolov8_vids.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --buildOnly --saveEngine=yolov8_vids.engine
[04/24/2023-12:38:53] [I] === Model Options ===
[04/24/2023-12:38:53] [I] Format: ONNX
[04/24/2023-12:38:53] [I] Model: best.onnx
[04/24/2023-12:38:53] [I] Output:
[04/24/2023-12:38:53] [I] === Build Options ===
[04/24/2023-12:38:53] [I] Max batch: explicit batch
[04/24/2023-12:38:53] [I] Workspace: 16 MiB
[04/24/2023-12:38:53] [I] minTiming: 1
[04/24/2023-12:38:53] [I] avgTiming: 8
[04/24/2023-12:38:53] [I] Precision: FP32+FP16
[04/24/2023-12:38:53] [I] Calibration:
[04/24/2023-12:38:53] [I] Refit: Disabled
[04/24/2023-12:38:53] [I] Sparsity: Disabled
[04/24/2023-12:38:53] [I] Safe mode: Disabled
[04/24/2023-12:38:53] [I] DirectIO mode: Disabled
[04/24/2023-12:38:53] [I] Restricted mode: Disabled
[04/24/2023-12:38:53] [I] Save engine: yolov8_vids.engine
[04/24/2023-12:38:53] [I] Load engine:
[04/24/2023-12:38:53] [I] Profiling verbosity: 0
[04/24/2023-12:38:53] [I] Tactic sources: Using default tactic sources
[04/24/2023-12:38:53] [I] timingCacheMode: local
[04/24/2023-12:38:53] [I] timingCacheFile:
[04/24/2023-12:38:53] [I] Input(s)s format: fp32:CHW
[04/24/2023-12:38:53] [I] Output(s)s format: fp32:CHW
[04/24/2023-12:38:53] [I] Input build shapes: model
[04/24/2023-12:38:53] [I] Input calibration shapes: model
[04/24/2023-12:38:53] [I] === System Options ===
[04/24/2023-12:38:53] [I] Device: 0
[04/24/2023-12:38:53] [I] DLACore:
[04/24/2023-12:38:53] [I] Plugins:
[04/24/2023-12:38:53] [I] === Inference Options ===
[04/24/2023-12:38:53] [I] Batch: Explicit
[04/24/2023-12:38:53] [I] Input inference shapes: model
[04/24/2023-12:38:53] [I] Iterations: 10
[04/24/2023-12:38:53] [I] Duration: 3s (+ 200ms warm up)
[04/24/2023-12:38:53] [I] Sleep time: 0ms
[04/24/2023-12:38:53] [I] Idle time: 0ms
[04/24/2023-12:38:53] [I] Streams: 1
[04/24/2023-12:38:53] [I] ExposeDMA: Disabled
[04/24/2023-12:38:53] [I] Data transfers: Enabled
[04/24/2023-12:38:53] [I] Spin-wait: Disabled
[04/24/2023-12:38:53] [I] Multithreading: Disabled
[04/24/2023-12:38:53] [I] CUDA Graph: Disabled
[04/24/2023-12:38:53] [I] Separate profiling: Disabled
[04/24/2023-12:38:53] [I] Time Deserialize: Disabled
[04/24/2023-12:38:53] [I] Time Refit: Disabled
[04/24/2023-12:38:53] [I] Skip inference: Enabled
[04/24/2023-12:38:53] [I] Inputs:
[04/24/2023-12:38:53] [I] === Reporting Options ===
[04/24/2023-12:38:53] [I] Verbose: Disabled
[04/24/2023-12:38:53] [I] Averages: 10 inferences
[04/24/2023-12:38:53] [I] Percentile: 99
[04/24/2023-12:38:53] [I] Dump refittable layers:Disabled
[04/24/2023-12:38:53] [I] Dump output: Disabled
[04/24/2023-12:38:53] [I] Profile: Disabled
[04/24/2023-12:38:53] [I] Export timing to JSON file:
[04/24/2023-12:38:53] [I] Export output to JSON file:
[04/24/2023-12:38:53] [I] Export profile to JSON file:
[04/24/2023-12:38:53] [I]
[04/24/2023-12:38:53] [I] === Device Information ===
[04/24/2023-12:38:53] [I] Selected Device: Xavier
[04/24/2023-12:38:53] [I] Compute Capability: 7.2
[04/24/2023-12:38:53] [I] SMs: 6
[04/24/2023-12:38:53] [I] Compute Clock Rate: 1.109 GHz
[04/24/2023-12:38:53] [I] Device Global Memory: 15825 MiB
[04/24/2023-12:38:53] [I] Shared Memory per SM: 96 KiB
[04/24/2023-12:38:53] [I] Memory Bus Width: 256 bits (ECC disabled)
[04/24/2023-12:38:53] [I] Memory Clock Rate: 1.109 GHz
[04/24/2023-12:38:53] [I]
[04/24/2023-12:38:53] [I] TensorRT version: 8.2.1
[04/24/2023-12:38:55] [I] [TRT] [MemUsageChange] Init CUDA: CPU +362, GPU +0, now: CPU 381, GPU 6567 (MiB)
[04/24/2023-12:38:56] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 381 MiB, GPU 6597 MiB
[04/24/2023-12:38:56] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 486 MiB, GPU 6726 MiB
[04/24/2023-12:38:56] [I] Start parsing network model
[04/24/2023-12:38:56] [I] [TRT] ----------------------------------------------------------------
[04/24/2023-12:38:56] [I] [TRT] Input filename: best.onnx
[04/24/2023-12:38:56] [I] [TRT] ONNX IR version: 0.0.6
[04/24/2023-12:38:56] [I] [TRT] Opset version: 11
[04/24/2023-12:38:56] [I] [TRT] Producer name: pytorch
[04/24/2023-12:38:56] [I] [TRT] Producer version: 2.0.0
[04/24/2023-12:38:56] [I] [TRT] Domain:
[04/24/2023-12:38:56] [I] [TRT] Model version: 0
[04/24/2023-12:38:56] [I] [TRT] Doc string:
[04/24/2023-12:38:56] [I] [TRT] ----------------------------------------------------------------
[04/24/2023-12:38:57] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[04/24/2023-12:38:57] [I] Finish parsing network model
[04/24/2023-12:38:57] [I] [TRT] ---------- Layers Running on DLA ----------
[04/24/2023-12:38:57] [I] [TRT] ---------- Layers Running on GPU ----------
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.0/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.0/act/Sigmoid), /model.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.1/act/Sigmoid), /model.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.2/cv1/act/Sigmoid), /model.2/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.2/m.0/cv1/act/Sigmoid), /model.2/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.2/m.0/cv2/act/Sigmoid), /model.2/m.0/cv2/act/Mul), /model.2/m.0/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.2/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.2/cv2/act/Sigmoid), /model.2/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.3/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.3/act/Sigmoid), /model.3/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.4/cv1/act/Sigmoid), /model.4/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.4/m.0/cv1/act/Sigmoid), /model.4/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.4/m.0/cv2/act/Sigmoid), /model.4/m.0/cv2/act/Mul), /model.4/m.0/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/m.1/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.4/m.1/cv1/act/Sigmoid), /model.4/m.1/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/m.1/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.4/m.1/cv2/act/Sigmoid), /model.4/m.1/cv2/act/Mul), /model.4/m.1/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/m.0/Add_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.4/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.4/cv2/act/Sigmoid), /model.4/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.5/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.5/act/Sigmoid), /model.5/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.6/cv1/act/Sigmoid), /model.6/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.6/m.0/cv1/act/Sigmoid), /model.6/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.6/m.0/cv2/act/Sigmoid), /model.6/m.0/cv2/act/Mul), /model.6/m.0/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/m.1/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.6/m.1/cv1/act/Sigmoid), /model.6/m.1/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/m.1/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.6/m.1/cv2/act/Sigmoid), /model.6/m.1/cv2/act/Mul), /model.6/m.1/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/m.0/Add_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.6/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.6/cv2/act/Sigmoid), /model.6/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.7/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.7/act/Sigmoid), /model.7/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.8/cv1/act/Sigmoid), /model.8/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.8/m.0/cv1/act/Sigmoid), /model.8/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(/model.8/m.0/cv2/act/Sigmoid), /model.8/m.0/cv2/act/Mul), /model.8/m.0/Add)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.8/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.8/cv2/act/Sigmoid), /model.8/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.9/cv1/act/Sigmoid), /model.9/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m/MaxPool
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m_1/MaxPool
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m_2/MaxPool
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/cv1/act/Mul_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m/MaxPool_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m_1/MaxPool_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/m_2/MaxPool_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.9/cv2/act/Sigmoid), /model.9/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.10/Resize
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.10/Resize_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.12/cv1/act/Sigmoid), /model.12/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.12/m.0/cv1/act/Sigmoid), /model.12/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.12/m.0/cv2/act/Sigmoid), /model.12/m.0/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.12/cv2/act/Sigmoid), /model.12/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.13/Resize
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.13/Resize_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.15/cv1/act/Sigmoid), /model.15/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.15/m.0/cv1/act/Sigmoid), /model.15/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.15/m.0/cv2/act/Sigmoid), /model.15/m.0/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.15/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.15/cv2/act/Sigmoid), /model.15/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.16/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.0/cv3.0.0/conv/Conv || /model.22/cv2.0/cv2.0.0/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.16/act/Sigmoid), /model.16/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.0/cv2.0.0/act/Sigmoid), /model.22/cv2.0/cv2.0.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.0/cv3.0.0/act/Sigmoid), /model.22/cv3.0/cv3.0.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.12/cv2/act/Mul_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.0/cv2.0.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.0/cv3.0.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.0/cv2.0.1/act/Sigmoid), /model.22/cv2.0/cv2.0.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.0/cv3.0.1/act/Sigmoid), /model.22/cv3.0/cv3.0.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.18/cv1/act/Sigmoid), /model.18/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.0/cv2.0.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.0/cv3.0.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.18/m.0/cv1/act/Sigmoid), /model.18/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.18/m.0/cv2/act/Sigmoid), /model.18/m.0/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.18/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.18/cv2/act/Sigmoid), /model.18/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.19/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.1/cv3.1.0/conv/Conv || /model.22/cv2.1/cv2.1.0/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.19/act/Sigmoid), /model.19/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.1/cv2.1.0/act/Sigmoid), /model.22/cv2.1/cv2.1.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.1/cv3.1.0/act/Sigmoid), /model.22/cv3.1/cv3.1.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.9/cv2/act/Mul_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.1/cv2.1.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.1/cv3.1.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.1/cv2.1.1/act/Sigmoid), /model.22/cv2.1/cv2.1.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.1/cv3.1.1/act/Sigmoid), /model.22/cv3.1/cv3.1.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.21/cv1/act/Sigmoid), /model.21/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.1/cv2.1.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.1/cv3.1.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/m.0/cv1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.21/m.0/cv1/act/Sigmoid), /model.21/m.0/cv1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/m.0/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.21/m.0/cv2/act/Sigmoid), /model.21/m.0/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/Split_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/Split_output_1 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.21/cv2/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.21/cv2/act/Sigmoid), /model.21/cv2/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.2/cv3.2.0/conv/Conv || /model.22/cv2.2/cv2.2.0/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.2/cv2.2.0/act/Sigmoid), /model.22/cv2.2/cv2.2.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.2/cv3.2.0/act/Sigmoid), /model.22/cv3.2/cv3.2.0/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.2/cv2.2.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.2/cv3.2.1/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv2.2/cv2.2.1/act/Sigmoid), /model.22/cv2.2/cv2.2.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(PWN(/model.22/cv3.2/cv3.2.1/act/Sigmoid), /model.22/cv3.2/cv3.2.1/act/Mul)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv2.2/cv2.2.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/cv3.2/cv3.2.2/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape_2
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape_1_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Reshape_2_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Split
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Split_8
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/dfl/Reshape + /model.22/dfl/Transpose
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 228) [Shuffle]
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/dfl/Softmax
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 230) [Shuffle] + /model.22/dfl/Transpose_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/dfl/conv/Conv
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/dfl/Reshape_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Slice
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Slice_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Constant_9_output_0
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Sub
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Constant_10_output_0
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Add_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(/model.22/Constant_11_output_0 + (Unnamed Layer* 292) [Shuffle], PWN(/model.22/Add_2, /model.22/Div_1))
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Sub_1
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Div_1_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Sub_1_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Constant_12_output_0 + (Unnamed Layer* 297) [Shuffle]
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Mul_2
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] PWN(/model.22/Sigmoid)
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Mul_2_output_0 copy
[04/24/2023-12:38:57] [I] [TRT] [GpuLayer] /model.22/Sigmoid_output_0 copy
[04/24/2023-12:38:58] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +227, GPU +324, now: CPU 763, GPU 7184 (MiB)
[04/24/2023-12:39:00] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +307, GPU +452, now: CPU 1070, GPU 7636 (MiB)
[04/24/2023-12:39:00] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[04/24/2023-12:39:00] [E] Error[2]: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{1794, 6, 16}, 12660},)
[04/24/2023-12:39:00] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[04/24/2023-12:39:00] [E] Engine could not be created from network
[04/24/2023-12:39:00] [E] Building engine failed
[04/24/2023-12:39:00] [E] Failed to create engine from model.
[04/24/2023-12:39:00] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # trtexec --onnx=best.onnx --fp16 --buildOnly --saveEngine=yolov8_vids.engine

AastaLLL · April 25, 2023, 5:02am

Hi,

Could you share the ONNX model with us so we can give it a check?
Thanks.

uditbajpai0495 · April 25, 2023, 9:05am

uditbajpai0495 · April 26, 2023, 7:16am

got the model?

AastaLLL · April 26, 2023, 7:58am

Hi,

Unknown embedded device detected. Please update the table with the entry: {{1794, 6, 16}, 12660},)

Are you using XavierNX 16GB?
There is a known issue in TensorRT on XavierNX 16GB.

Please set up your environment with JetPack 4.6.3 or JetPack 5 to get the fix.
Confirm that we can run your model with JetPack 4.6.3 on Xavier.

...
[04/26/2023-15:43:53] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +226, GPU +344, now: CPU 762, GPU 4139 (MiB)
[04/26/2023-15:43:55] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +308, GPU +518, now: CPU 1070, GPU 4657 (MiB)
[04/26/2023-15:43:55] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[04/26/2023-15:47:25] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.

[04/26/2023-15:56:34] [I] [TRT] Detected 1 inputs and 3 output network tensors.
[04/26/2023-15:56:34] [I] [TRT] Total Host Persistent Memory: 154688
[04/26/2023-15:56:34] [I] [TRT] Total Device Persistent Memory: 22415872
[04/26/2023-15:56:34] [I] [TRT] Total Scratch Memory: 1075200
[04/26/2023-15:56:34] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 105 MiB
[04/26/2023-15:56:34] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 51.8052ms to assign 8 blocks to 147 nodes requiring 18432001 bytes.
[04/26/2023-15:56:34] [I] [TRT] Total Activation Memory: 18432001
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1563, GPU 6654 (MiB)
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1563, GPU 6654 (MiB)
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +21, GPU +32, now: CPU 21, GPU 32 (MiB)
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1551, GPU 6679 (MiB)
[04/26/2023-15:56:34] [I] [TRT] Loaded engine size: 24 MiB
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1562, GPU 6679 (MiB)
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1562, GPU 6679 (MiB)
[04/26/2023-15:56:34] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +21, now: CPU 0, GPU 21 (MiB)
[04/26/2023-15:56:34] [I] Engine built in 765.598 sec.
&&&& PASSED TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=best.onnx --fp16 --buildOnly

Thanks.

uditbajpai0495 · April 26, 2023, 12:26pm

Thanks

system · May 17, 2023, 12:48am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
ERORR with ONNX2TRT : Unknown embedded device detected Jetson Xavier NX onnx	18	4501	April 27, 2022
Model onnx trt engine generation process report different results compared between PC and jetson XAVIER NX Jetson Xavier NX tensorrt	19	1006	September 28, 2022
About trtexec Jetson Nano tensorrt	2	3352	October 15, 2021
Trtexec convert onnx to engine fails TAO Toolkit	14	1063	October 30, 2023
I do not get any performance improvement after using TensorRT provider for object detection model Jetson Nano tensorrt , onnx	7	1381	July 12, 2022
DLA performance DeepStream SDK	17	107	September 23, 2024
Erorr with onnx to trt Jetson Xavier NX tensorrt	8	1231	March 30, 2022
Unable to build model engine for INT8 yolov8m quantized using tensorrt model optimizer TensorRT jetson , deepstream	5	179	September 24, 2024
Error loading .trt model Jetson AGX Orin tensorrt	7	71	November 6, 2024
Onnx to trt conversion TensorRT tensorrt	8	783	April 21, 2020

Trying to convert Yolov8.onnx into trt ( TensorRT version : 8.2, jetson-jetpack : 4.6.1)

Related topics