Orin trtexec results in Segmentation Fault

manbehindthemadness · February 13, 2023, 7:52pm

I am attempting to convert the RobusBackgroundMatting (GitHub - PeterL1n/RobustVideoMatting: Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!) network into TensorRT. I am basing my procedure on the following: TensorRT 开始 - GoCodingInMyWay - 博客园
In addition, to build onnxruntime I referenced this: Issue using Onnxruntime with CUDAExecutionProvider on Orin

R35 (release), REVISION: 1.0, GCID: 31346300, BOARD: t186ref, EABI: aarch64, DATE: Thu Aug 25 18:41:45 UTC 2022

Reproducible example (just run convert.sh):
note that the script opens two images to confirm inference within onnx is operational, these need to be closed to procede
onnx_trt_utils.zip (2.5 MB)

Error:

onnx_trt_utils$ ./convert.sh

--2023-02-13 14:43:14--  https://github.com/PeterL1n/RobustVideoMatting/releases/download/v1.0.0/rvm_mobilenetv3_fp32.onnx
Resolving github.com (github.com)... 140.82.112.4
Connecting to github.com (github.com)|140.82.112.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/401484223/dae3f0eb-04b1-4f0d-b4b0-806275f206be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230213T194315Z&X-Amz-Expires=300&X-Amz-Signature=d93dd3e5f1520e6008fa39e0f98137818418bf8a7020392ba7aab18ff39db1de&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401484223&response-content-disposition=attachment%3B%20filename%3Drvm_mobilenetv3_fp32.onnx&response-content-type=application%2Foctet-stream [following]
--2023-02-13 14:43:15--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/401484223/dae3f0eb-04b1-4f0d-b4b0-806275f206be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230213T194315Z&X-Amz-Expires=300&X-Amz-Signature=d93dd3e5f1520e6008fa39e0f98137818418bf8a7020392ba7aab18ff39db1de&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401484223&response-content-disposition=attachment%3B%20filename%3Drvm_mobilenetv3_fp32.onnx&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.108.133, 185.199.111.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14975696 (14M) [application/octet-stream]
Saving to: ‘rvm_mobilenetv3_fp32.onnx’

rvm_mobilenetv3_fp32.onnx       100%[======================================================>]  14.28M  8.23MB/s    in 1.7s

2023-02-13 14:43:17 (8.23 MB/s) - ‘rvm_mobilenetv3_fp32.onnx’ saved [14975696/14975696]

Args
  model: rvm_mobilenetv3_fp32.onnx
  input_image: input.jpg
  input_resize: None
  output_image: input_com.png
  precision: Precision.FLOAT32
  show: True
save to input_com.png
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃                   ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add               │ 16             │ 16               │
│ AveragePool       │ 3              │ 3                │
│ Clip              │ 2              │ 2                │
│ Concat            │ 23             │ 23               │
│ Constant          │ 48             │ 0                │
│ Conv              │ 85             │ 85               │
│ Div               │ 1              │ 1                │
│ Expand            │ 4              │ 4                │
│ GlobalAveragePool │ 9              │ 9                │
│ HardSigmoid       │ 28             │ 28               │
│ Mul               │ 47             │ 47               │
│ ReduceMean        │ 2              │ 2                │
│ Relu              │ 27             │ 27               │
│ Resize            │ 7              │ 7                │
│ Shape             │ 12             │ 10               │
│ Sigmoid           │ 5              │ 5                │
│ Slice             │ 12             │ 10               │
│ Split             │ 10             │ 10               │
│ Sub               │ 8              │ 8                │
│ Tanh              │ 4              │ 4                │
│ Model Size        │ 14.3MiB        │ 14.3MiB          │
└───────────────────┴────────────────┴──────────────────┘
Args
  input: rvm_mobilenetv3_fp32_sim.onnx
  output: rvm_mobilenetv3_fp32_sim_modified.onnx
  downsample_ratio: 0.2
  input_size: [2560, 1440]

onnx load: rvm_mobilenetv3_fp32_sim.onnx

graph.opset=12

graph.inputs
  Variable (src): (shape=[1, 3, 1440, 2560], dtype=float32)
  Variable (r1i): (shape=[1, 1, 1, 1], dtype=float32)
  Variable (r2i): (shape=[1, 1, 1, 1], dtype=float32)
  Variable (r3i): (shape=[1, 1, 1, 1], dtype=float32)
  Variable (r4i): (shape=[1, 1, 1, 1], dtype=float32)
  Variable (downsample_ratio): (shape=[1], dtype=float32)

graph.outputs
  Variable (fgr): (shape=['batch_size', 3, 'height', 'width'], dtype=float32)
  Variable (pha): (shape=['batch_size', 1, 'height', 'width'], dtype=float32)
  Variable (r1o): (shape=['batch_size', 16, 'height', 'width'], dtype=float32)
  Variable (r2o): (shape=['batch_size', 20, 'height', 'width'], dtype=float32)
  Variable (r3o): (shape=['batch_size', 40, 'height', 'width'], dtype=float32)
  Variable (r4o): (shape=['batch_size', 64, 'height', 'width'], dtype=float32)

Resize_3 (Resize)
        Inputs: [
                Variable (src): (shape=[1, 3, 1440, 2560], dtype=float32)
                Constant (386): (shape=[0], dtype=<class 'numpy.float32'>)
                Variable (388): (shape=[4], dtype=float32)
        ]
        Outputs: [
                Variable (389): (shape=['unk__0', 'unk__1', 'unk__2', 'unk__3'], dtype=float32)
        ]
Attributes: OrderedDict([('coordinate_transformation_mode', 'pytorch_half_pixel'), ('cubic_coeff_a', -0.75), ('mode', 'linear'), ('nearest_mode', 'floor')])

Resize_3 (Resize)
        Inputs: [
                Variable (src): (shape=[1, 3, 1440, 2560], dtype=float32)
                Constant (386): (shape=[0], dtype=<class 'numpy.float32'>)
                Constant (388): (shape=(4,), dtype=<class 'numpy.float32'>)
        ]
        Outputs: [
                Variable (389): (shape=['unk__0', 'unk__1', 'unk__2', 'unk__3'], dtype=float32)
        ]
Attributes: OrderedDict([('coordinate_transformation_mode', 'pytorch_half_pixel'), ('cubic_coeff_a', -0.75), ('mode', 'linear'), ('nearest_mode', 'floor')])
  INFO    | onnx-typecast | ONNX INT64 --> INT32 Converter | (convert.py:71)
  INFO    | onnx-typecast | Loading Model: rvm_mobilenetv3_fp32_sim_modified.onnx | (convert.py:72)
  INFO    | onnx-typecast | Converting INT64 model params to INT32... | (convert.py:83)
  INFO    | onnx-typecast | Converting constant INT64 nodes to INT32... | (convert.py:86)
  INFO    | onnx-typecast | Creating new graph... | (convert.py:90)
  INFO    | onnx-typecast | Creating new int32 model... | (convert.py:99)
  INFO    | onnx-typecast | Saving converted model as: rvm_mobilenetv3_fp32_sim_modified_fp32.onnx | (convert.py:103)
  INFO    | onnx-typecast | Done Done London. 🎉 | (convert.py:105)
&&&& RUNNING TensorRT.trtexec [TensorRT v8503] # trtexec --onnx=rvm_mobilenetv3_fp32_sim_modified_fp32.onnx --workspace=64 --saveEngine=rvm_mobilenetv3_fp32_sim_modified.engine --verbose
[02/13/2023-14:43:27] [W] --workspace flag has been deprecated by --memPoolSize flag.
[02/13/2023-14:43:27] [I] === Model Options ===
[02/13/2023-14:43:27] [I] Format: ONNX
[02/13/2023-14:43:27] [I] Model: rvm_mobilenetv3_fp32_sim_modified_fp32.onnx
[02/13/2023-14:43:27] [I] Output:
[02/13/2023-14:43:27] [I] === Build Options ===
[02/13/2023-14:43:27] [I] Max batch: explicit batch
[02/13/2023-14:43:27] [I] Memory Pools: workspace: 64 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[02/13/2023-14:43:27] [I] minTiming: 1
[02/13/2023-14:43:27] [I] avgTiming: 8
[02/13/2023-14:43:27] [I] Precision: FP32
[02/13/2023-14:43:27] [I] LayerPrecisions:
[02/13/2023-14:43:27] [I] Calibration:
[02/13/2023-14:43:27] [I] Refit: Disabled
[02/13/2023-14:43:27] [I] Sparsity: Disabled
[02/13/2023-14:43:27] [I] Safe mode: Disabled
[02/13/2023-14:43:27] [I] DirectIO mode: Disabled
[02/13/2023-14:43:27] [I] Restricted mode: Disabled
[02/13/2023-14:43:27] [I] Build only: Disabled
[02/13/2023-14:43:27] [I] Save engine: rvm_mobilenetv3_fp32_sim_modified.engine
[02/13/2023-14:43:27] [I] Load engine:
[02/13/2023-14:43:27] [I] Profiling verbosity: 0
[02/13/2023-14:43:27] [I] Tactic sources: Using default tactic sources
[02/13/2023-14:43:27] [I] timingCacheMode: local
[02/13/2023-14:43:27] [I] timingCacheFile:
[02/13/2023-14:43:27] [I] Heuristic: Disabled
[02/13/2023-14:43:27] [I] Preview Features: Use default preview flags.
[02/13/2023-14:43:27] [I] Input(s)s format: fp32:CHW
[02/13/2023-14:43:27] [I] Output(s)s format: fp32:CHW
[02/13/2023-14:43:27] [I] Input build shapes: model
[02/13/2023-14:43:27] [I] Input calibration shapes: model
[02/13/2023-14:43:27] [I] === System Options ===
[02/13/2023-14:43:27] [I] Device: 0
[02/13/2023-14:43:27] [I] DLACore:
[02/13/2023-14:43:27] [I] Plugins:
[02/13/2023-14:43:27] [I] === Inference Options ===
[02/13/2023-14:43:27] [I] Batch: Explicit
[02/13/2023-14:43:27] [I] Input inference shapes: model
[02/13/2023-14:43:27] [I] Iterations: 10
[02/13/2023-14:43:27] [I] Duration: 3s (+ 200ms warm up)
[02/13/2023-14:43:27] [I] Sleep time: 0ms
[02/13/2023-14:43:27] [I] Idle time: 0ms
[02/13/2023-14:43:27] [I] Streams: 1
[02/13/2023-14:43:27] [I] ExposeDMA: Disabled
[02/13/2023-14:43:27] [I] Data transfers: Enabled
[02/13/2023-14:43:27] [I] Spin-wait: Disabled
[02/13/2023-14:43:27] [I] Multithreading: Disabled
[02/13/2023-14:43:27] [I] CUDA Graph: Disabled
[02/13/2023-14:43:27] [I] Separate profiling: Disabled
[02/13/2023-14:43:27] [I] Time Deserialize: Disabled
[02/13/2023-14:43:27] [I] Time Refit: Disabled
[02/13/2023-14:43:27] [I] NVTX verbosity: 0
[02/13/2023-14:43:27] [I] Persistent Cache Ratio: 0
[02/13/2023-14:43:27] [I] Inputs:
[02/13/2023-14:43:27] [I] === Reporting Options ===
[02/13/2023-14:43:27] [I] Verbose: Enabled
[02/13/2023-14:43:27] [I] Averages: 10 inferences
[02/13/2023-14:43:27] [I] Percentiles: 90,95,99
[02/13/2023-14:43:27] [I] Dump refittable layers:Disabled
[02/13/2023-14:43:27] [I] Dump output: Disabled
[02/13/2023-14:43:27] [I] Profile: Disabled
[02/13/2023-14:43:27] [I] Export timing to JSON file:
[02/13/2023-14:43:27] [I] Export output to JSON file:
[02/13/2023-14:43:27] [I] Export profile to JSON file:
[02/13/2023-14:43:27] [I]
[02/13/2023-14:43:27] [I] === Device Information ===
[02/13/2023-14:43:27] [I] Selected Device: Orin
[02/13/2023-14:43:27] [I] Compute Capability: 8.7
[02/13/2023-14:43:27] [I] SMs: 16
[02/13/2023-14:43:27] [I] Compute Clock Rate: 1.3 GHz
[02/13/2023-14:43:27] [I] Device Global Memory: 30535 MiB
[02/13/2023-14:43:27] [I] Shared Memory per SM: 164 KiB
[02/13/2023-14:43:27] [I] Memory Bus Width: 128 bits (ECC disabled)
[02/13/2023-14:43:27] [I] Memory Clock Rate: 1.3 GHz
[02/13/2023-14:43:27] [I]
[02/13/2023-14:43:27] [I] TensorRT version: 8.5.3
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Proposal version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::Split version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[02/13/2023-14:43:27] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[02/13/2023-14:43:28] [I] [TRT] [MemUsageChange] Init CUDA: CPU +218, GPU +0, now: CPU 242, GPU 5716 (MiB)
[02/13/2023-14:43:32] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +351, GPU +330, now: CPU 612, GPU 6064 (MiB)
[02/13/2023-14:43:32] [I] Start parsing network model
[02/13/2023-14:43:32] [I] [TRT] ----------------------------------------------------------------
[02/13/2023-14:43:32] [I] [TRT] Input filename:   rvm_mobilenetv3_fp32_sim_modified_fp32.onnx
[02/13/2023-14:43:32] [I] [TRT] ONNX IR version:  0.0.8
[02/13/2023-14:43:32] [I] [TRT] Opset version:    12
[02/13/2023-14:43:32] [I] [TRT] Producer name:    onnx-typecast
[02/13/2023-14:43:32] [I] [TRT] Producer version:
[02/13/2023-14:43:32] [I] [TRT] Domain:
[02/13/2023-14:43:32] [I] [TRT] Model version:    0
[02/13/2023-14:43:32] [I] [TRT] Doc string:
[02/13/2023-14:43:32] [I] [TRT] ----------------------------------------------------------------
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::Split version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[02/13/2023-14:43:32] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[02/13/2023-14:43:32] [V] [TRT] Adding network input: src with dtype: float32, dimensions: (1, 3, 1440, 2560)
[02/13/2023-14:43:32] [V] [TRT] Registering tensor: src for ONNX tensor: src
[02/13/2023-14:43:32] [V] [TRT] Adding network input: r1i with dtype: float32, dimensions: (1, 1, 1, 1)
[02/13/2023-14:43:32] [V] [TRT] Registering tensor: r1i for ONNX tensor: r1i
[02/13/2023-14:43:32] [V] [TRT] Adding network input: r2i with dtype: float32, dimensions: (1, 1, 1, 1)
[02/13/2023-14:43:32] [V] [TRT] Registering tensor: r2i for ONNX tensor: r2i
[02/13/2023-14:43:32] [V] [TRT] Adding network input: r3i with dtype: float32, dimensions: (1, 1, 1, 1)
[02/13/2023-14:43:32] [V] [TRT] Registering tensor: r3i for ONNX tensor: r3i
[02/13/2023-14:43:32] [V] [TRT] Adding network input: r4i with dtype: float32, dimensions: (1, 1, 1, 1)
[02/13/2023-14:43:32] [V] [TRT] Registering tensor: r4i for ONNX tensor: r4i
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 386
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 388
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 390
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 392
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 824
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 825
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 827
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 828
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 830
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 831
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 833
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 834
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 836
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 837
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 839
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 840
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 842
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 843
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 845
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 846
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 848
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 849
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 851
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 852
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 854
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 855
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.4.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.4.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.4.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.4.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 857
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 858
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 860
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 861
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 863
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 864
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.5.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.5.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.5.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.5.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 866
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 867
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 869
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 870
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 872
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 873
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.6.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.6.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.6.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.6.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 875
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 876
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 878
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 879
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 881
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 882
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 884
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 885
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 887
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 888
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 890
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 891
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 893
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 894
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 896
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 897
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 899
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 900
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 902
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 903
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 905
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 906
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 908
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 909
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 911
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 912
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 914
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 915
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 917
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 918
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.11.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.11.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.11.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.11.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 920
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 921
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 923
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 924
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 926
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 927
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.12.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.12.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.12.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.12.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 929
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 930
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 932
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 933
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 935
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 936
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.13.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.13.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.13.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.13.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 938
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 939
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 941
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 942
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 944
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 945
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.14.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.14.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.14.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.14.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 947
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 948
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 950
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 951
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 953
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 954
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.15.block.2.fc1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.15.block.2.fc1.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.15.block.2.fc2.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: backbone.features.15.block.2.fc2.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 956
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 957
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 959
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 960
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 962
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 963
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: aspp.aspp2.1.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode4.gru.ih.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode4.gru.ih.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode4.gru.hh.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode4.gru.hh.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 990
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 985
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 630
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 631
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 632
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 634
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 635
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 965
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 966
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode3.gru.ih.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode3.gru.ih.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode3.gru.hh.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode3.gru.hh.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 968
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 969
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode2.gru.ih.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode2.gru.ih.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode2.gru.hh.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode2.gru.hh.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 971
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 972
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode1.gru.ih.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode1.gru.ih.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode1.gru.hh.0.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: decoder.decode1.gru.hh.0.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 974
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 975
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 977
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 978
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: project_mat.conv.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: project_mat.conv.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: refiner.box_filter.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 980
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 981
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 983
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 984
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: refiner.conv.6.weight
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: refiner.conv.6.bias
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 815
[02/13/2023-14:43:32] [V] [TRT] Importing initializer: 989
[02/13/2023-14:43:32] [V] [TRT] Parsing node: Resize_3 [Resize]
[02/13/2023-14:43:32] [V] [TRT] Searching for input: src
[02/13/2023-14:43:32] [V] [TRT] Searching for input: 386
[02/13/2023-14:43:32] [V] [TRT] Searching for input: 388
[02/13/2023-14:43:32] [V] [TRT] Resize_3 [Resize] inputs: [src -> (1, 3, 1440, 2560)[FLOAT]], [386 -> (0)[FLOAT]], [388 -> (4)[FLOAT]],
[02/13/2023-14:43:32] [V] [TRT] Registering layer: Resize_3 for ONNX node: Resize_3
./convert.sh: line 32: 113668 Segmentation fault      (core dumped) trtexec --onnx=rvm_mobilenetv3_fp32_sim_modified_fp32.onnx --workspace=64 --saveEngine=rvm_mobilenetv3_fp32_sim_modified.engine --verbose

What am I doing wrong here?

manbehindthemadness · February 14, 2023, 1:11am

[UPDATE]
After reinstalling TensorRT and the remainder of the libraries paying close attention to matching the versions and adjusting the environment variables, I now get through the compilation process; however, this procedure fails on deserialization (again segmentation fault):

trtexec --verbose:
error.txt (3.5 MB)

polygraphy -vv:
polygraphy.txt (145.7 KB)

AastaLLL · February 14, 2023, 2:28am

Hi,

Thanks for reporting this.

We are trying to reproduce this issue with your source.
Will update more information later.

AastaLLL · February 14, 2023, 8:47am

Hi,

We meet the following error when testing the model:

$ /usr/src/tensorrt/bin/trtexec --onnx=rvm_mobilenetv3_fp32.onnx
&&&& RUNNING TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --onnx=rvm_mobilenetv3_fp32.onnx
...
[02/14/2023-08:35:55] [E] Error[4]: [graphShapeAnalyzer.cpp::demandAndResolveInputTensor::1246] Error Code 4: Internal Error (downsample_ratio: network input that is shape tensor must have type Int32)
[02/14/2023-08:35:55] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3 [Resize -> "389"]:
[02/14/2023-08:35:55] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[02/14/2023-08:35:55] [E] [TRT] ModelImporter.cpp:728: input: "src"
input: "386"
input: "388"
output: "389"
name: "Resize_3"
op_type: "Resize"
attribute {
  name: "coordinate_transformation_mode"
  s: "pytorch_half_pixel"
  type: STRING
}
attribute {
  name: "cubic_coeff_a"
  f: -0.75
  type: FLOAT
}
attribute {
  name: "mode"
  s: "linear"
  type: STRING
}
attribute {
  name: "nearest_mode"
  s: "floor"
  type: STRING
}

[02/14/2023-08:35:55] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[02/14/2023-08:35:55] [E] [TRT] ModelImporter.cpp:731: ERROR: ModelImporter.cpp:185 In function parseGraph:
[6] Invalid Node - Resize_3
[graphShapeAnalyzer.cpp::demandAndResolveInputTensor::1246] Error Code 4: Internal Error (downsample_ratio: network input that is shape tensor must have type Int32)
...

It looks like the model contains some non-supported layers.
Please check the requirement of resize layer below:
https://github.com/onnx/onnx-tensorrt/blob/main/docs/operators.md

Thanks.

manbehindthemadness · February 14, 2023, 12:20pm

Yes, we freeze that in the conversion process and fix the downsample_ratio variable on line 25 of the conversion script using convert.py. Then we ensure all the values are int32 on line 28 before passing the output to trtexec.

manbehindthemadness · February 15, 2023, 6:07pm

After some work I loaded the engine that is created from this process and performed test inference successfully (the segmentation fault still occurs after compile when using trtexec, but the engine file is created)

This being said it would appear that the issue is in some form of post-operation check and not in the compile itself.

AastaLLL · February 16, 2023, 6:42am

Hi,

Good to know this.
So the inference can work with your application, is this correct?

Thanks.

manbehindthemadness · February 16, 2023, 12:31pm

The example code works proving the engine functions, I am still adopting the logic to my application.

manbehindthemadness · February 16, 2023, 3:38pm

It appears that everything functions; however, the outputs in python are problematic. This is probably due to a mistake on my part, I will address that in a separate topic. Thanks for the help.

AastaLLL · February 17, 2023, 1:00am

Thanks for the info.

system · March 3, 2023, 1:01am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Trtexec conversion to int8 Jetson AGX Orin tensorrt	2	983	October 11, 2023
Encountering error while trying to convert an ONNX model to TensorRT engine - Jetson Orin Nano TensorRT tensorrt , pytorch , cudnn , onnx , jetson	1	82	July 23, 2024
ONNX to TensorRT conversion Jetson Nano tensorrt , pytorch , onnx	4	322	July 3, 2024
PyTorch/ONNX Model Involving Very Large Images: Myelin error: autotuning: CUDA error 2 allocating 0-byte buffer: out of memory TensorRT pytorch	4	1019	February 9, 2023
Trtexec cannot convert QAT onnx model to trt model TensorRT tensorrt	7	1140	January 26, 2023
How to export the Pytorch model Keypoint R-CNN to onnx and benchmark with trtexec TensorRT	7	1329	July 14, 2022
Trtexec "Failed to sort the model topologically." TensorRT tensorrt , cudnn , onnx	2	559	March 5, 2024
Duplicated reshapes triggers "[graphOptimizer.cpp::findOne::510] Error Code 2: Internal Error (Assertion it != v.end() failed. )" TensorRT tensorrt , onnx	4	918	February 22, 2022
Parse onnx file failed: Parameter check failed. condition:condition: allDimsGtEq(windowSize, 1) && volume(windowSize) < MAX_KERNEL_DIMS_PRODUCT(nbSpat TensorRT tensorrt	3	944	May 17, 2022
Floor - Cast - Resize(or Slice) cause internal error TensorRT tensorrt	7	1999	January 12, 2022

Orin trtexec results in Segmentation Fault

Related topics