Environment
TensorRT Version: 8.2.3
GPU Type: A30
Nvidia Driver Version: 525.147.05
CUDA Version: 12
CUDNN Version:
Operating System + Version: Ubuntu 20
Python Version (if applicable): 3.8.12
PyTorch Version (if applicable): 1.12.0a0+2c916ef
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/pytorch:22.03-py3
The model file is here.
onnx file has opset14.
When conversion is done with the following command,
trtexec --onnx=vae.onnx --saveEngine=vae.plan --minShapes=latent_sample:1x4x64x64 --optShapes=latent_sample:4x4x64x64 --maxShapes=latent_sample:8x4x64x64 --fp16
I have errors as
Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"
The whole error log is as follows.
root@72095ff6d485:/mount# trtexec --onnx=vae.onnx --saveEngine=vae.plan --minShapes=latent_sample:1x4x64x64 --optShapes=latent_sample:4x4x64x64 --maxShapes=latent_sample:8x4x64x64 --fp16
&&&& RUNNING TensorRT.trtexec [TensorRT v8203] # trtexec --onnx=vae.onnx --saveEngine=vae.plan --minShapes=latent_sample:1x4x64x64 --optShapes=latent_sample:4x4x64x64 --maxShapes=latent_sample:8x4x64x64 --fp16
[12/05/2023-01:10:02] [I] === Model Options ===
[12/05/2023-01:10:02] [I] Format: ONNX
[12/05/2023-01:10:02] [I] Model: vae.onnx
[12/05/2023-01:10:02] [I] Output:
[12/05/2023-01:10:02] [I] === Build Options ===
[12/05/2023-01:10:02] [I] Max batch: explicit batch
[12/05/2023-01:10:02] [I] Workspace: 16 MiB
[12/05/2023-01:10:02] [I] minTiming: 1
[12/05/2023-01:10:02] [I] avgTiming: 8
[12/05/2023-01:10:02] [I] Precision: FP32+FP16
[12/05/2023-01:10:02] [I] Calibration:
[12/05/2023-01:10:02] [I] Refit: Disabled
[12/05/2023-01:10:02] [I] Sparsity: Disabled
[12/05/2023-01:10:02] [I] Safe mode: Disabled
[12/05/2023-01:10:02] [I] DirectIO mode: Disabled
[12/05/2023-01:10:02] [I] Restricted mode: Disabled
[12/05/2023-01:10:02] [I] Save engine: vae.plan
[12/05/2023-01:10:02] [I] Load engine:
[12/05/2023-01:10:02] [I] Profiling verbosity: 0
[12/05/2023-01:10:02] [I] Tactic sources: Using default tactic sources
[12/05/2023-01:10:02] [I] timingCacheMode: local
[12/05/2023-01:10:02] [I] timingCacheFile:
[12/05/2023-01:10:02] [I] Input(s)s format: fp32:CHW
[12/05/2023-01:10:02] [I] Output(s)s format: fp32:CHW
[12/05/2023-01:10:02] [I] Input build shape: latent_sample=1x4x64x64+4x4x64x64+8x4x64x64
[12/05/2023-01:10:02] [I] Input calibration shapes: model
[12/05/2023-01:10:02] [I] === System Options ===
[12/05/2023-01:10:02] [I] Device: 0
[12/05/2023-01:10:02] [I] DLACore:
[12/05/2023-01:10:02] [I] Plugins:
[12/05/2023-01:10:02] [I] === Inference Options ===
[12/05/2023-01:10:02] [I] Batch: Explicit
[12/05/2023-01:10:02] [I] Input inference shape: latent_sample=4x4x64x64
[12/05/2023-01:10:02] [I] Iterations: 10
[12/05/2023-01:10:02] [I] Duration: 3s (+ 200ms warm up)
[12/05/2023-01:10:02] [I] Sleep time: 0ms
[12/05/2023-01:10:02] [I] Idle time: 0ms
[12/05/2023-01:10:02] [I] Streams: 1
[12/05/2023-01:10:02] [I] ExposeDMA: Disabled
[12/05/2023-01:10:02] [I] Data transfers: Enabled
[12/05/2023-01:10:02] [I] Spin-wait: Disabled
[12/05/2023-01:10:02] [I] Multithreading: Disabled
[12/05/2023-01:10:02] [I] CUDA Graph: Disabled
[12/05/2023-01:10:02] [I] Separate profiling: Disabled
[12/05/2023-01:10:02] [I] Time Deserialize: Disabled
[12/05/2023-01:10:02] [I] Time Refit: Disabled
[12/05/2023-01:10:02] [I] Skip inference: Disabled
[12/05/2023-01:10:02] [I] Inputs:
[12/05/2023-01:10:02] [I] === Reporting Options ===
[12/05/2023-01:10:02] [I] Verbose: Disabled
[12/05/2023-01:10:02] [I] Averages: 10 inferences
[12/05/2023-01:10:02] [I] Percentile: 99
[12/05/2023-01:10:02] [I] Dump refittable layers:Disabled
[12/05/2023-01:10:02] [I] Dump output: Disabled
[12/05/2023-01:10:02] [I] Profile: Disabled
[12/05/2023-01:10:02] [I] Export timing to JSON file:
[12/05/2023-01:10:02] [I] Export output to JSON file:
[12/05/2023-01:10:02] [I] Export profile to JSON file:
[12/05/2023-01:10:02] [I]
[12/05/2023-01:10:02] [I] === Device Information ===
[12/05/2023-01:10:02] [I] Selected Device: NVIDIA A30
[12/05/2023-01:10:02] [I] Compute Capability: 8.0
[12/05/2023-01:10:02] [I] SMs: 56
[12/05/2023-01:10:02] [I] Compute Clock Rate: 1.44 GHz
[12/05/2023-01:10:02] [I] Device Global Memory: 24059 MiB
[12/05/2023-01:10:02] [I] Shared Memory per SM: 164 KiB
[12/05/2023-01:10:02] [I] Memory Bus Width: 3072 bits (ECC enabled)
[12/05/2023-01:10:02] [I] Memory Clock Rate: 1.215 GHz
[12/05/2023-01:10:02] [I]
[12/05/2023-01:10:02] [I] TensorRT version: 8.2.3
[12/05/2023-01:10:03] [I] [TRT] [MemUsageChange] Init CUDA: CPU +428, GPU +0, now: CPU 440, GPU 471 (MiB)
[12/05/2023-01:10:04] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 440 MiB, GPU 471 MiB
[12/05/2023-01:10:04] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 657 MiB, GPU 543 MiB
[12/05/2023-01:10:04] [I] Start parsing network model
[12/05/2023-01:10:04] [I] [TRT] ----------------------------------------------------------------
[12/05/2023-01:10:04] [I] [TRT] Input filename: vae.onnx
[12/05/2023-01:10:04] [I] [TRT] ONNX IR version: 0.0.7
[12/05/2023-01:10:04] [I] [TRT] Opset version: 14
[12/05/2023-01:10:04] [I] [TRT] Producer name: pytorch
[12/05/2023-01:10:04] [I] [TRT] Producer version: 1.12.0
[12/05/2023-01:10:04] [I] [TRT] Domain:
[12/05/2023-01:10:04] [I] [TRT] Model version: 0
[12/05/2023-01:10:04] [I] [TRT] Doc string:
[12/05/2023-01:10:04] [I] [TRT] ----------------------------------------------------------------
[12/05/2023-01:10:04] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/05/2023-01:10:06] [E] [TRT] parsers/onnx/ModelImporter.cpp:780: While parsing node number 301 [Resize -> "input.204"]:
[12/05/2023-01:10:06] [E] [TRT] parsers/onnx/ModelImporter.cpp:781: --- Begin node ---
[12/05/2023-01:10:06] [E] [TRT] parsers/onnx/ModelImporter.cpp:782: input: "hidden_states.3"
input: ""
input: "onnx::Resize_910"
output: "input.204"
name: "Resize_301"
op_type: "Resize"
attribute {
name: "coordinate_transformation_mode"
s: "asymmetric"
type: STRING
}
attribute {
name: "cubic_coeff_a"
f: -0.75
type: FLOAT
}
attribute {
name: "mode"
s: "nearest"
type: STRING
}
attribute {
name: "nearest_mode"
s: "floor"
type: STRING
}
[12/05/2023-01:10:06] [E] [TRT] parsers/onnx/ModelImporter.cpp:783: --- End node ---
[12/05/2023-01:10:06] [E] [TRT] parsers/onnx/ModelImporter.cpp:785: ERROR: parsers/onnx/builtin_op_importers.cpp:3608 In function importResize:
[8] Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"
[12/05/2023-01:10:06] [E] Failed to parse onnx file
[12/05/2023-01:10:06] [I] Finish parsing network model
[12/05/2023-01:10:06] [E] Parsing model failed
[12/05/2023-01:10:06] [E] Failed to create engine from model.
[12/05/2023-01:10:06] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8203]