Help Needed. Unable to run VideoMAE model on Xavier NX

oakvnnn · November 19, 2024, 4:13pm

Hey all,

I am trying to run VideoMAE on Jetson Xavier NX. However, I keep getting an unsupported operator error when trying to convert it to TensorRT. I have the model download locally in ONNX format. Please note that CUDA, pycuda, torch, and everything necessary is already set up.
Error:

Failed to parse the ONNX file.
In node 13 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

Code:

import tensorrt as trt
import pycuda.driver as cuda
import pycuda.autoinit

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

def build_engine(onnx_file_path, engine_file_path):
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
        # Create a BuilderConfig object
        config = builder.create_builder_config()
        config.max_workspace_size = 1 << 30  # 1GB

        # Enable INT8 mode if supported
        if builder.platform_has_fast_int8:
            config.set_flag(trt.BuilderFlag.INT8)

        # Parse the ONNX model
        with open(onnx_file_path, 'rb') as model:
            if not parser.parse(model.read()):
                print("Failed to parse the ONNX file.")
                for error in range(parser.num_errors):
                    print(parser.get_error(error))
                return None

        # Build the engine
        engine = builder.build_engine(network, config)
        if engine is None:
            print("Failed to build the engine.")
            return None

        # Serialize the engine to a file
        with open(engine_file_path, "wb") as f:
            f.write(engine.serialize())

build_engine("test_model.onnx", "test_model.trt")

Part of jtop output:

Libraries:
CUDA: 11.4.315
cuDNN: 8.6.0.166
TensorRT: 8.5.2.2
VPI: 2.2.4
Vulkan: 1.3.204
OpenCV: 4.5.4 with CUDA: NO

SivaRamaKrishnaNV · November 21, 2024, 12:57pm

Dear @oakvnnn,
Could you check if the model has unsupported layers? Can you quickly test using trtexec tool ?

oakvnnn · November 21, 2024, 2:06pm

@SivaRamaKrishnaNV Thank you for the reply.
It still does not work with the trtexec tool. Does that mean there is no way of running it?
Below is the output:

11/21/2024-14:04:03] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[11/21/2024-14:04:03] [E] [TRT] ModelImporter.cpp:731: ERROR: builtin_op_importers.cpp:5427 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[11/21/2024-14:04:03] [E] Failed to parse onnx file
[11/21/2024-14:04:03] [I] Finish parsing network model
[11/21/2024-14:04:03] [E] Parsing model failed
[11/21/2024-14:04:03] [E] Failed to create engine from model or file.
[11/21/2024-14:04:03] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=test_model.onnx --saveEngine=test_model.trt --explicitBatch --int8

SivaRamaKrishnaNV · November 21, 2024, 2:34pm

Dear @oakvnnn,
It seems model has unsupported layers. Could you share the ONNX model and trtexec complete log with --verbose flag.

oakvnnn · November 21, 2024, 3:36pm

@SivaRamaKrishnaNV
Here is the ONNX model in Google Drive.
Here is the complete log of trtexec:

&&&& RUNNING TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=test_model.onnx --saveEngine=test_model.trt --explicitBatch --int8 --verbose
[11/21/2024-15:32:08] [W] --explicitBatch flag has been deprecated and has no effect!
[11/21/2024-15:32:08] [W] Explicit batch dim is automatically enabled if input model is ONNX or if dynamic shapes are provided when the engine is built.
[11/21/2024-15:32:08] [I] === Model Options ===
[11/21/2024-15:32:08] [I] Format: ONNX
[11/21/2024-15:32:08] [I] Model: test_model.onnx
[11/21/2024-15:32:08] [I] Output:
[11/21/2024-15:32:08] [I] === Build Options ===
[11/21/2024-15:32:08] [I] Max batch: explicit batch
[11/21/2024-15:32:08] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[11/21/2024-15:32:08] [I] minTiming: 1
[11/21/2024-15:32:08] [I] avgTiming: 8
[11/21/2024-15:32:08] [I] Precision: FP32+INT8
[11/21/2024-15:32:08] [I] LayerPrecisions: 
[11/21/2024-15:32:08] [I] Calibration: Dynamic
[11/21/2024-15:32:08] [I] Refit: Disabled
[11/21/2024-15:32:08] [I] Sparsity: Disabled
[11/21/2024-15:32:08] [I] Safe mode: Disabled
[11/21/2024-15:32:08] [I] DirectIO mode: Disabled
[11/21/2024-15:32:08] [I] Restricted mode: Disabled
[11/21/2024-15:32:08] [I] Build only: Disabled
[11/21/2024-15:32:08] [I] Save engine: test_model.trt
[11/21/2024-15:32:08] [I] Load engine: 
[11/21/2024-15:32:08] [I] Profiling verbosity: 0
[11/21/2024-15:32:08] [I] Tactic sources: Using default tactic sources
[11/21/2024-15:32:08] [I] timingCacheMode: local
[11/21/2024-15:32:08] [I] timingCacheFile: 
[11/21/2024-15:32:08] [I] Heuristic: Disabled
[11/21/2024-15:32:08] [I] Preview Features: Use default preview flags.
[11/21/2024-15:32:08] [I] Input(s)s format: fp32:CHW
[11/21/2024-15:32:08] [I] Output(s)s format: fp32:CHW
[11/21/2024-15:32:08] [I] Input build shapes: model
[11/21/2024-15:32:08] [I] Input calibration shapes: model
[11/21/2024-15:32:08] [I] === System Options ===
[11/21/2024-15:32:08] [I] Device: 0
[11/21/2024-15:32:08] [I] DLACore: 
[11/21/2024-15:32:08] [I] Plugins:
[11/21/2024-15:32:08] [I] === Inference Options ===
[11/21/2024-15:32:08] [I] Batch: Explicit
[11/21/2024-15:32:08] [I] Input inference shapes: model
[11/21/2024-15:32:08] [I] Iterations: 10
[11/21/2024-15:32:08] [I] Duration: 3s (+ 200ms warm up)
[11/21/2024-15:32:08] [I] Sleep time: 0ms
[11/21/2024-15:32:08] [I] Idle time: 0ms
[11/21/2024-15:32:08] [I] Streams: 1
[11/21/2024-15:32:08] [I] ExposeDMA: Disabled
[11/21/2024-15:32:08] [I] Data transfers: Enabled
[11/21/2024-15:32:08] [I] Spin-wait: Disabled
[11/21/2024-15:32:08] [I] Multithreading: Disabled
[11/21/2024-15:32:08] [I] CUDA Graph: Disabled
[11/21/2024-15:32:08] [I] Separate profiling: Disabled
[11/21/2024-15:32:08] [I] Time Deserialize: Disabled
[11/21/2024-15:32:08] [I] Time Refit: Disabled
[11/21/2024-15:32:08] [I] NVTX verbosity: 0
[11/21/2024-15:32:08] [I] Persistent Cache Ratio: 0
[11/21/2024-15:32:08] [I] Inputs:
[11/21/2024-15:32:08] [I] === Reporting Options ===
[11/21/2024-15:32:08] [I] Verbose: Enabled
[11/21/2024-15:32:08] [I] Averages: 10 inferences
[11/21/2024-15:32:08] [I] Percentiles: 90,95,99
[11/21/2024-15:32:08] [I] Dump refittable layers:Disabled
[11/21/2024-15:32:08] [I] Dump output: Disabled
[11/21/2024-15:32:08] [I] Profile: Disabled
[11/21/2024-15:32:08] [I] Export timing to JSON file: 
[11/21/2024-15:32:08] [I] Export output to JSON file: 
[11/21/2024-15:32:08] [I] Export profile to JSON file: 
[11/21/2024-15:32:08] [I] 
[11/21/2024-15:32:08] [I] === Device Information ===
[11/21/2024-15:32:08] [I] Selected Device: Xavier
[11/21/2024-15:32:08] [I] Compute Capability: 7.2
[11/21/2024-15:32:08] [I] SMs: 6
[11/21/2024-15:32:08] [I] Compute Clock Rate: 1.109 GHz
[11/21/2024-15:32:08] [I] Device Global Memory: 6848 MiB
[11/21/2024-15:32:08] [I] Shared Memory per SM: 96 KiB
[11/21/2024-15:32:08] [I] Memory Bus Width: 256 bits (ECC disabled)
[11/21/2024-15:32:08] [I] Memory Clock Rate: 0.51 GHz
[11/21/2024-15:32:08] [I] 
[11/21/2024-15:32:08] [I] TensorRT version: 8.5.2
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::GroupNorm version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::LayerNorm version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Proposal version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::SeqLen2Spatial version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::SplitGeLU version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::Split version 1
[11/21/2024-15:32:08] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[11/21/2024-15:32:16] [I] [TRT] [MemUsageChange] Init CUDA: CPU +187, GPU +0, now: CPU 213, GPU 2984 (MiB)
[11/21/2024-15:32:16] [V] [TRT] Trying to load shared library libnvinfer_builder_resource.so.8.5.2
[11/21/2024-15:32:16] [V] [TRT] Loaded shared library libnvinfer_builder_resource.so.8.5.2
[11/21/2024-15:32:18] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +106, GPU +149, now: CPU 342, GPU 3156 (MiB)
[11/21/2024-15:32:19] [I] Start parsing network model
[11/21/2024-15:32:23] [I] [TRT] ----------------------------------------------------------------
[11/21/2024-15:32:23] [I] [TRT] Input filename:   test_model.onnx
[11/21/2024-15:32:23] [I] [TRT] ONNX IR version:  0.0.8
[11/21/2024-15:32:23] [I] [TRT] Opset version:    17
[11/21/2024-15:32:23] [I] [TRT] Producer name:    pytorch
[11/21/2024-15:32:23] [I] [TRT] Producer version: 2.5.0
[11/21/2024-15:32:23] [I] [TRT] Domain:           
[11/21/2024-15:32:23] [I] [TRT] Model version:    0
[11/21/2024-15:32:23] [I] [TRT] Doc string:       
[11/21/2024-15:32:23] [I] [TRT] ----------------------------------------------------------------
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::GroupNorm version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::LayerNorm version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::ROIAlign_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::SeqLen2Spatial version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::SplitGeLU version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::Split version 1
[11/21/2024-15:32:24] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[11/21/2024-15:32:24] [V] [TRT] Adding network input: input with dtype: float32, dimensions: (1, 16, 3, 224, 224)
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: input for ONNX tensor: input
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.embeddings.patch_embeddings.projection.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.embeddings.patch_embeddings.projection.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.0.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.1.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.2.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.3.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.4.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.5.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.6.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.7.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.8.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.9.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.10.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.attention.attention.q_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.attention.attention.v_bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.attention.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.intermediate.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.output.dense.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.layernorm_after.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.videomae.encoder.layer.11.layernorm_after.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.fc_norm.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.fc_norm.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.classifier.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: videomae.classifier.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: classifier.weight
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: classifier.bias
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1374
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1375
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1376
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1396
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1397
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1398
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1399
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1400
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1401
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1421
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1422
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1423
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1424
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1425
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1426
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1446
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1447
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1448
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1449
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1450
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1451
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1471
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1472
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1473
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1474
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1475
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1476
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1496
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1497
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1498
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1499
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1500
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1501
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1521
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1522
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1523
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1524
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1525
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1526
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1546
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1547
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1548
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1549
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1550
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1551
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1571
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1572
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1573
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1574
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1575
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1576
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1596
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1597
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1598
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1599
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1600
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1601
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1621
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1622
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1623
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1624
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1625
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1626
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1646
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1647
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1648
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1649
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1650
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1651
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1671
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1672
[11/21/2024-15:32:24] [V] [TRT] Importing initializer: onnx::MatMul_1673
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Transpose [Transpose]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: input
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Transpose [Transpose] inputs: [input -> (1, 16, 3, 224, 224)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Transpose for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Transpose
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Transpose_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Transpose_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Transpose [Transpose] outputs: [/videomae/videomae/embeddings/patch_embeddings/Transpose_output_0 -> (1, 3, 16, 224, 224)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/projection/Conv [Conv]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Transpose_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: videomae.videomae.embeddings.patch_embeddings.projection.weight
[11/21/2024-15:32:24] [V] [TRT] Searching for input: videomae.videomae.embeddings.patch_embeddings.projection.bias
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/projection/Conv [Conv] inputs: [/videomae/videomae/embeddings/patch_embeddings/Transpose_output_0 -> (1, 3, 16, 224, 224)[FLOAT]], [videomae.videomae.embeddings.patch_embeddings.projection.weight -> (768, 3, 2, 16, 16)[FLOAT]], [videomae.videomae.embeddings.patch_embeddings.projection.bias -> (768)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Convolution input dimensions: (1, 3, 16, 224, 224)
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/projection/Conv for ONNX node: /videomae/videomae/embeddings/patch_embeddings/projection/Conv
[11/21/2024-15:32:24] [V] [TRT] Using kernel: (2, 16, 16), strides: (2, 16, 16), prepadding: (0, 0, 0), postpadding: (0, 0, 0), dilations: (1, 1, 1), numOutputs: 768
[11/21/2024-15:32:24] [V] [TRT] Convolution output dimensions: (1, 768, 8, 14, 14)
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/projection/Conv [Conv] outputs: [/videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0 -> (1, 768, 8, 14, 14)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Shape [Shape]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Shape [Shape] inputs: [/videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0 -> (1, 768, 8, 14, 14)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Shape for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Shape
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Shape_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Shape_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Shape [Shape] outputs: [/videomae/videomae/embeddings/patch_embeddings/Shape_output_0 -> (5)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Constant [Constant]
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant [Constant] inputs: 
[11/21/2024-15:32:24] [W] [TRT] onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant [Constant] outputs: [/videomae/videomae/embeddings/patch_embeddings/Constant_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Constant_1 [Constant]
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_1 [Constant] inputs: 
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_1 [Constant] outputs: [/videomae/videomae/embeddings/patch_embeddings/Constant_1_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Constant_2 [Constant]
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_2 [Constant] inputs: 
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_2 [Constant] outputs: [/videomae/videomae/embeddings/patch_embeddings/Constant_2_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Slice [Slice]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Shape_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Constant_1_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Constant_2_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Constant_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Slice [Slice] inputs: [/videomae/videomae/embeddings/patch_embeddings/Shape_output_0 -> (5)[INT32]], [/videomae/videomae/embeddings/patch_embeddings/Constant_1_output_0 -> (1)[INT32]], [/videomae/videomae/embeddings/patch_embeddings/Constant_2_output_0 -> (1)[INT32]], [/videomae/videomae/embeddings/patch_embeddings/Constant_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Slice for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Slice
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Slice_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Slice_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Slice [Slice] outputs: [/videomae/videomae/embeddings/patch_embeddings/Slice_output_0 -> (2)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Constant_3 [Constant]
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_3 [Constant] inputs: 
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Constant_3 [Constant] outputs: [/videomae/videomae/embeddings/patch_embeddings/Constant_3_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Concat [Concat]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Slice_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Constant_3_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Concat [Concat] inputs: [/videomae/videomae/embeddings/patch_embeddings/Slice_output_0 -> (2)[INT32]], [/videomae/videomae/embeddings/patch_embeddings/Constant_3_output_0 -> (1)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Constant_3_output_0 for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Constant_3_output_0
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Concat for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Concat
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Concat_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Concat_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Concat [Concat] outputs: [/videomae/videomae/embeddings/patch_embeddings/Concat_output_0 -> (3)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Reshape [Reshape]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Concat_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Reshape [Reshape] inputs: [/videomae/videomae/embeddings/patch_embeddings/projection/Conv_output_0 -> (1, 768, 8, 14, 14)[FLOAT]], [/videomae/videomae/embeddings/patch_embeddings/Concat_output_0 -> (3)[INT32]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Reshape for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Reshape
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Reshape_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Reshape_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Reshape [Reshape] outputs: [/videomae/videomae/embeddings/patch_embeddings/Reshape_output_0 -> (1, 768, 1568)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/patch_embeddings/Transpose_1 [Transpose]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Reshape_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Transpose_1 [Transpose] inputs: [/videomae/videomae/embeddings/patch_embeddings/Reshape_output_0 -> (1, 768, 1568)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/patch_embeddings/Transpose_1 for ONNX node: /videomae/videomae/embeddings/patch_embeddings/Transpose_1
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/patch_embeddings/Transpose_1_output_0 for ONNX tensor: /videomae/videomae/embeddings/patch_embeddings/Transpose_1_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/patch_embeddings/Transpose_1 [Transpose] outputs: [/videomae/videomae/embeddings/patch_embeddings/Transpose_1_output_0 -> (1, 1568, 768)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/Constant [Constant]
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/Constant [Constant] inputs: 
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/Constant [Constant] outputs: [/videomae/videomae/embeddings/Constant_output_0 -> (1, 1568, 768)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/embeddings/Add [Add]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/patch_embeddings/Transpose_1_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/Constant_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/Add [Add] inputs: [/videomae/videomae/embeddings/patch_embeddings/Transpose_1_output_0 -> (1, 1568, 768)[FLOAT]], [/videomae/videomae/embeddings/Constant_output_0 -> (1, 1568, 768)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/Constant_output_0 for ONNX node: /videomae/videomae/embeddings/Constant_output_0
[11/21/2024-15:32:24] [V] [TRT] Registering layer: /videomae/videomae/embeddings/Add for ONNX node: /videomae/videomae/embeddings/Add
[11/21/2024-15:32:24] [V] [TRT] Registering tensor: /videomae/videomae/embeddings/Add_output_0 for ONNX tensor: /videomae/videomae/embeddings/Add_output_0
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/embeddings/Add [Add] outputs: [/videomae/videomae/embeddings/Add_output_0 -> (1, 1568, 768)[FLOAT]], 
[11/21/2024-15:32:24] [V] [TRT] Parsing node: /videomae/videomae/encoder/layer.0/layernorm_before/LayerNormalization [LayerNormalization]
[11/21/2024-15:32:24] [V] [TRT] Searching for input: /videomae/videomae/embeddings/Add_output_0
[11/21/2024-15:32:24] [V] [TRT] Searching for input: videomae.videomae.encoder.layer.0.layernorm_before.weight
[11/21/2024-15:32:24] [V] [TRT] Searching for input: videomae.videomae.encoder.layer.0.layernorm_before.bias
[11/21/2024-15:32:24] [V] [TRT] /videomae/videomae/encoder/layer.0/layernorm_before/LayerNormalization [LayerNormalization] inputs: [/videomae/videomae/embeddings/Add_output_0 -> (1, 1568, 768)[FLOAT]], [videomae.videomae.encoder.layer.0.layernorm_before.weight -> (768)[FLOAT]], [videomae.videomae.encoder.layer.0.layernorm_before.bias -> (768)[FLOAT]], 
[11/21/2024-15:32:24] [I] [TRT] No importer registered for op: LayerNormalization. Attempting to import as plugin.
[11/21/2024-15:32:24] [I] [TRT] Searching for plugin: LayerNormalization, plugin_version: 1, plugin_namespace: 
[11/21/2024-15:32:24] [E] [TRT] ModelImporter.cpp:726: While parsing node number 13 [LayerNormalization -> "/videomae/videomae/encoder/layer.0/layernorm_before/LayerNormalization_output_0"]:
[11/21/2024-15:32:24] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[11/21/2024-15:32:24] [E] [TRT] ModelImporter.cpp:728: input: "/videomae/videomae/embeddings/Add_output_0"
input: "videomae.videomae.encoder.layer.0.layernorm_before.weight"
input: "videomae.videomae.encoder.layer.0.layernorm_before.bias"
output: "/videomae/videomae/encoder/layer.0/layernorm_before/LayerNormalization_output_0"
name: "/videomae/videomae/encoder/layer.0/layernorm_before/LayerNormalization"
op_type: "LayerNormalization"
attribute {
  name: "axis"
  i: -1
  type: INT
}
attribute {
  name: "epsilon"
  f: 1e-12
  type: FLOAT
}
doc_string: "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py(2900): layer_norm\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/normalization.py(217): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/transformers/models/videomae/modeling_videomae.py(426): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/transformers/models/videomae/modeling_videomae.py(481): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/transformers/models/videomae/modeling_videomae.py(697): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/transformers/models/videomae/modeling_videomae.py(1086): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n<ipython-input-7-f48474c856c0>(9): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1726): _slow_forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/torch/jit/_trace.py(130): wrapper\n/usr/local/lib/python3.10/dist-packages/torch/jit/_trace.py(139): forward\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1747): _call_impl\n/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl\n/usr/local/lib/python3.10/dist-packages/torch/jit/_trace.py(1500): _get_trace_graph\n/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py(904): _trace_and_get_graph_from_model\n/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py(997): _create_jit_graph\n/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py(1113): _model_to_graph\n/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py(1564): _export\n/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py(502): export\n/usr/local/lib/python3.10/dist-packages/torch/onnx/__init__.py(375): export\n<ipython-input-14-5054be50dc5d>(16): <cell line: 16>\n/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py(3553): run_code\n/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py(3473): run_ast_nodes\n/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py(3257): run_cell_async\n/usr/local/lib/python3.10/dist-packages/IPython/core/async_helpers.py(78): _pseudo_sync_runner\n/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py(3030): _run_cell\n/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py(2975): run_cell\n/usr/local/lib/python3.10/dist-packages/ipykernel/zmqshell.py(539): run_cell\n/usr/local/lib/python3.10/dist-packages/ipykernel/ipkernel.py(302): do_execute\n/usr/local/lib/python3.10/dist-packages/tornado/gen.py(234): wrapper\n/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py(539): execute_request\n/usr/local/lib/python3.10/dist-packages/tornado/gen.py(234): wrapper\n/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py(261): dispatch_shell\n/usr/local/lib/python3.10/dist-packages/tornado/gen.py(234): wrapper\n/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py(361): process_one\n/usr/local/lib/python3.10/dist-packages/tornado/gen.py(786): run\n/usr/local/lib/python3.10/dist-packages/tornado/gen.py(825): inner\n/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py(738): _run_callback\n/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py(685): <lambda>\n/usr/lib/python3.10/asyncio/events.py(80): _run\n/usr/lib/python3.10/asyncio/base_events.py(1909): _run_once\n/usr/lib/python3.10/asyncio/base_events.py(603): run_forever\n/usr/local/lib/python3.10/dist-packages/tornado/platform/asyncio.py(195): start\n/usr/local/lib/python3.10/dist-packages/ipykernel/kernelapp.py(619): start\n/usr/local/lib/python3.10/dist-packages/traitlets/config/application.py(992): launch_instance\n/usr/local/lib/python3.10/dist-packages/colab_kernel_launcher.py(37): <module>\n/usr/lib/python3.10/runpy.py(86): _run_code\n/usr/lib/python3.10/runpy.py(196): _run_module_as_main\n"

[11/21/2024-15:32:24] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[11/21/2024-15:32:24] [E] [TRT] ModelImporter.cpp:731: ERROR: builtin_op_importers.cpp:5427 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[11/21/2024-15:32:24] [E] Failed to parse onnx file
[11/21/2024-15:32:24] [I] Finish parsing network model
[11/21/2024-15:32:24] [E] Parsing model failed
[11/21/2024-15:32:24] [E] Failed to create engine from model or file.
[11/21/2024-15:32:24] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=test_model.onnx --saveEngine=test_model.trt --explicitBatch --int8 --verbose

SivaRamaKrishnaNV · November 25, 2024, 9:47am

The support for LayerNormalization is added from TRT 8.6.1 as per Release Notes :: NVIDIA Deep Learning TensorRT Documentation . But the Jetpack 5.1.4 has TRT 8.5.2 . Can you try using LayerNorm plugin?

oakvnnn · November 26, 2024, 12:39pm

Thank you @SivaRamaKrishnaNV . Could you provide a reference to the plugin?

SivaRamaKrishnaNV · December 3, 2024, 4:14pm

Dear @oakvnnn ,
Please check Developer Guide :: NVIDIA Deep Learning TensorRT Documentation .

system · January 1, 2025, 12:42am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Encountering error while trying to convert an ONNX model to TensorRT engine - Jetson Orin Nano TensorRT tensorrt , pytorch , cudnn , onnx , jetson	1	101	July 23, 2024
Trtexec "Failed to sort the model topologically." TensorRT tensorrt , cudnn , onnx	2	641	March 5, 2024
Trtexec cannot convert QAT onnx model to trt model TensorRT tensorrt	7	1195	January 26, 2023
Trtexec Error: only activation types allowed as input to this layer TensorRT tensorrt	8	1527	June 28, 2024
ONNX to TensorRT conversion Jetson Nano tensorrt , pytorch , onnx	4	378	July 3, 2024
Parse onnx file failed: Parameter check failed. condition:condition: allDimsGtEq(windowSize, 1) && volume(windowSize) < MAX_KERNEL_DIMS_PRODUCT(nbSpat TensorRT tensorrt	3	953	May 17, 2022
Run out of memory when creating TensorRT engine from onnx model Jetson Xavier NX tensorrt	7	2776	October 18, 2021
Trtexec conversion to int8 Jetson AGX Orin tensorrt	2	1126	October 11, 2023
[TensorRT] ERROR: Network must have at least one output [TensorRT] ERROR: Network validation failed TensorRT tensorrt , cuda , onnx	10	2790	October 16, 2020
How to export the Pytorch model Keypoint R-CNN to onnx and benchmark with trtexec TensorRT	7	1356	July 14, 2022

Help Needed. Unable to run VideoMAE model on Xavier NX

Related topics