Run out of memory when creating TensorRT engine from onnx model

lvkaleo · June 17, 2020, 11:02pm

I have a custom PyTorch model, which I exported to onnx format (1.3 gb)
Now I wanted to run it on Jetson Xavier NX using TensorRT.
When trying to create an trt engine (similar to the yolov3 example provided on Jetson Xavier Nx in /usr/src/tensorrt/samples/python/yolov3_onnx/onnx_to_tensorrt.py) I’m getting the following errors:

[TensorRT] ERROR: Try increasing the workspace size with IBuilderConfig::setMaxWorkspaceSize() if using IBuilder::buildEngineWithConfig, or IBuilder::setMaxWorkspaceSize() if using IBuilder::buildCudaEngine.
[TensorRT] ERROR: ../builder/tacticOptimizer.cpp (1715) - TRTInternal Error in computeCosts: 0 (Could not find any implementation for node {(Unnamed Layer* 116) [Constant],(Unnamed Layer* 117) [Constant],(Unnamed Layer* 125) [Constant],(Unnamed Layer* 65) [Shuffle] + (Unnamed Layer* 77) [Shuffle],(Unnamed Layer* 73) [Constant] + (Unnamed Layer* 74) [Shuffle],(Unnamed Layer* 75) [Slice],(Unnamed Layer* 96) [Slice],(Unnamed Layer* 115) [Slice],(Unnamed Layer* 126) [TripLimit],(Unnamed Layer* 127) [Iterator],(Unnamed Layer* 128) [Shuffle],(Unnamed Layer* 131) [Matrix Multiply],(Unnamed Layer* 129) [Recurrence],(Unnamed Layer* 130) [Recurrence],(Unnamed Layer* 132) [Matrix Multiply],(Unnamed Layer* 133) [ElementWise],(Unnamed Layer* 134) [ElementWise],(Unnamed Layer* 135) [Slice],(Unnamed Layer* 138) [Slice],(Unnamed Layer* 141) [Slice],(Unnamed Layer* 147) [Slice],(Unnamed Layer* 137) [Activation],(Unnamed Layer* 140) [Activation],(Unnamed Layer* 143) [Activation],(Unnamed Layer* 149) [Activation],(Unnamed Layer* 144) [ElementWise],(Unnamed Layer* 145) [ElementWise],(Unnamed Layer* 146) [ElementWise],(Unnamed Layer* 150) [Activation],(Unnamed Layer* 151) [ElementWise],(Unnamed Layer* 155) [LoopOutput],(Unnamed Layer* 213) [Gather]_output[Constant],(Unnamed Layer* 200) [Constant],(Unnamed Layer* 201) [Constant],(Unnamed Layer* 180) [Slice],(Unnamed Layer* 199) [Slice],(Unnamed Layer* 161) [Shuffle],(Unnamed Layer* 222) [Recurrence],(Unnamed Layer* 223) [Recurrence],(Unnamed Layer* 225) [Matrix Multiply],(Unnamed Layer* 215) [Iterator],(Unnamed Layer* 214) [TripLimit],(Unnamed Layer* 221) [Shuffle],(Unnamed Layer* 224) [Matrix Multiply],(Unnamed Layer* 226) [ElementWise],(Unnamed Layer* 227) [ElementWise],(Unnamed Layer* 228) [Slice],(Unnamed Layer* 231) [Slice],(Unnamed Layer* 234) [Slice],(Unnamed Layer* 240) [Slice],(Unnamed Layer* 230) [Activation],(Unnamed Layer* 233) [Activation],(Unnamed Layer* 236) [Activation],(Unnamed Layer* 242) [Activation],(Unnamed Layer* 237) [ElementWise],(Unnamed Layer* 238) [ElementWise],(Unnamed Layer* 239) [ElementWise],(Unnamed Layer* 243) [Activation],(Unnamed Layer* 244) [ElementWise],(Unnamed Layer* 250) [LoopOutput],(Unnamed Layer* 256) [Shuffle] + (Unnamed Layer* 257) [Shuffle],(Unnamed Layer* 258) [Constant] + (Unnamed Layer* 259) [Shuffle],(Unnamed Layer* 260) [Matrix Multiply],(Unnamed Layer* 261) [Constant] + (Unnamed Layer* 262) [Shuffle],(Unnamed Layer* 263) [ElementWise]}.)
[TensorRT] ERROR: ../builder/tacticOptimizer.cpp (1715) - TRTInternal Error in computeCosts: 0 ()

Additionally, I’m getting the following warnings:

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:81] The total number of bytes read was 1335373289
[TensorRT] WARNING: onnx2trt_utils.cpp:217: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:243: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.

Checking the onnx model using onnx.checker passed. The memory consumption for running the onnx model is ~ 4.5 GB.

I also tried converting my model using the trtexec tool:

/usr/src/tensorrt/bin/trtexec --onnx=model.onnx --shapes=input:1x6x3x278x920 --maxBatch=1 --workspace=4800 --verbose

which resulted in the process getting killed by the system:

&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=model.onnx --shapes=input:1x6x3x278x920 --maxBatch=1 --workspace=4800 --verbose
[06/17/2020-16:34:21] [I] === Model Options ===
[06/17/2020-16:34:21] [I] Format: ONNX
[06/17/2020-16:34:21] [I] Model: model.onnx
[06/17/2020-16:34:21] [I] Output:
[06/17/2020-16:34:21] [I] === Build Options ===
[06/17/2020-16:34:21] [I] Max batch: explicit
[06/17/2020-16:34:21] [I] Workspace: 4800 MB
[06/17/2020-16:34:21] [I] minTiming: 1
[06/17/2020-16:34:21] [I] avgTiming: 8
[06/17/2020-16:34:21] [I] Precision: FP32
[06/17/2020-16:34:21] [I] Calibration: 
[06/17/2020-16:34:21] [I] Safe mode: Disabled
[06/17/2020-16:34:21] [I] Save engine: 
[06/17/2020-16:34:21] [I] Load engine: 
[06/17/2020-16:34:21] [I] Builder Cache: Enabled
[06/17/2020-16:34:21] [I] NVTX verbosity: 0
[06/17/2020-16:34:21] [I] Inputs format: fp32:CHW
[06/17/2020-16:34:21] [I] Outputs format: fp32:CHW
[06/17/2020-16:34:21] [I] Input build shape: input=1x6x3x278x920+1x6x3x278x920+1x6x3x278x920
[06/17/2020-16:34:21] [I] Input calibration shapes: model
[06/17/2020-16:34:21] [I] === System Options ===
[06/17/2020-16:34:21] [I] Device: 0
[06/17/2020-16:34:21] [I] DLACore: 
[06/17/2020-16:34:21] [I] Plugins:
[06/17/2020-16:34:21] [I] === Inference Options ===
[06/17/2020-16:34:21] [I] Batch: Explicit
[06/17/2020-16:34:21] [I] Input inference shape: input=1x6x3x278x920
[06/17/2020-16:34:21] [I] Iterations: 10
[06/17/2020-16:34:21] [I] Duration: 3s (+ 200ms warm up)
[06/17/2020-16:34:21] [I] Sleep time: 0ms
[06/17/2020-16:34:21] [I] Streams: 1
[06/17/2020-16:34:21] [I] ExposeDMA: Disabled
[06/17/2020-16:34:21] [I] Spin-wait: Disabled
[06/17/2020-16:34:21] [I] Multithreading: Disabled
[06/17/2020-16:34:21] [I] CUDA Graph: Disabled
[06/17/2020-16:34:21] [I] Skip inference: Disabled
[06/17/2020-16:34:21] [I] Inputs:
[06/17/2020-16:34:21] [I] === Reporting Options ===
[06/17/2020-16:34:21] [I] Verbose: Enabled
[06/17/2020-16:34:21] [I] Averages: 10 inferences
[06/17/2020-16:34:21] [I] Percentile: 99
[06/17/2020-16:34:21] [I] Dump output: Disabled
[06/17/2020-16:34:21] [I] Profile: Disabled
[06/17/2020-16:34:21] [I] Export timing to JSON file: 
[06/17/2020-16:34:21] [I] Export output to JSON file: 
[06/17/2020-16:34:21] [I] Export profile to JSON file: 
[06/17/2020-16:34:21] [I] 
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::GridAnchor_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::NMS_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Reorg_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Region_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Clip_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::LReLU_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::PriorBox_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Normalize_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::RPROI_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::BatchedNMS_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::FlattenConcat_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::CropAndResize
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::DetectionLayer_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Proposal
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::ProposalLayer_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::PyramidROIAlign_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::ResizeNearest_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::Split
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::SpecialSlice_TRT
[06/17/2020-16:34:21] [V] [TRT] Plugin creator registration succeeded - ::InstanceNormalization_TRT
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:81] The total number of bytes read was 1335373289
----------------------------------------------------------------
Input filename:   deepvo.onnx
ONNX IR version:  0.0.4
Opset version:    10
Producer name:    pytorch
Producer version: 1.3
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:81] The total number of bytes read was 1335373289
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::NMS_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Reorg_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Region_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Clip_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::LReLU_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Normalize_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::RPROI_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::CropAndResize
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Proposal
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::Split
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT
[06/17/2020-16:34:46] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:205: Adding network input: input with dtype: float32, dimensions: (-1, 6, 3, 278, 920)
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:97: Registering tensor: input for ONNX tensor: input
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 252
[06/17/2020-16:34:46] [W] [TRT] onnx2trt_utils.cpp:217: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 253
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 271
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 272
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 273
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 291
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 292
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 293
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 294
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv1.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv1.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv1.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv1.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv1.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3_1.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3_1.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3_1.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3_1.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv3_1.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4_1.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4_1.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4_1.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4_1.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv4_1.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5_1.0.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5_1.1.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5_1.1.running_mean
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5_1.1.running_var
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv5_1.1.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv6.weight
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: linear.bias
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [60 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [61 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [62 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [63 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: input
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 61
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 62
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 60
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 63
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [input -> (-1, 6, 3, 278, 920)], [61 -> (1)], [62 -> (1)], [60 -> (1)], [63 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 0) [Shape] for ONNX node: 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 64 for ONNX tensor: 64
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [64 -> (-1, -1, -1, -1, -1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [65 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [66 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] onnx2trt_utils.cpp:233: Weight at index 0: 9223372036854775807 is out of range. Clamping to: 2147483647
[06/17/2020-16:34:46] [W] [TRT] onnx2trt_utils.cpp:243: One or more weights outside the range of INT32 was clamped
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [67 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [68 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: input
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 66
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 67
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 65
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 68
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [input -> (-1, 6, 3, 278, 920)], [66 -> (1)], [67 -> (1)], [65 -> (1)], [68 -> (1)], 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 19) [Shape] for ONNX node: 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 69 for ONNX tensor: 69
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [69 -> (-1, -1, -1, -1, -1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Concat]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 64
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 69
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Concat] inputs: [64 -> (-1, -1, -1, -1, -1)], [69 -> (-1, -1, -1, -1, -1)], 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 38) [Concatenation] for ONNX node: 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 70 for ONNX tensor: 70
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Concat] outputs: [70 -> (-1, -1, -1, -1, -1)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [71 -> (4)], 
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Reshape]
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 70
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 71
[06/17/2020-16:34:46] [V] [TRT] ModelImporter.cpp:125:  [Reshape] inputs: [70 -> (-1, -1, -1, -1, -1)], [71 -> (4)], 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 39) [Shuffle] for ONNX node: 
[06/17/2020-16:34:46] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 72 for ONNX tensor: 72
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Reshape] outputs: [72 -> (80, 6, 278, 920)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 72
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv1.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [72 -> (80, 6, 278, 920)], [conv1.0.weight -> (64, 6, 7, 7)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 6, 278, 920)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (7, 7), strides: (2, 2), prepadding: (3, 3), postpadding: (3, 3), dilations: (1, 1), numOutputs: 64
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 64, 139, 460)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 40) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 73 for ONNX tensor: 73
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [73 -> (80, 64, 139, 460)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 73
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv1.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv1.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv1.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv1.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [73 -> (80, 64, 139, 460)], [conv1.1.weight -> (64)], [conv1.1.bias -> (64)], [conv1.1.running_mean -> (64)], [conv1.1.running_var -> (64)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 41) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 74 for ONNX tensor: 74
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [74 -> (80, 64, 139, 460)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 74
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [74 -> (80, 64, 139, 460)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 42) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 75 for ONNX tensor: 75
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [75 -> (80, 64, 139, 460)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 75
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [75 -> (80, 64, 139, 460)], [conv2.0.weight -> (128, 64, 5, 5)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 64, 139, 460)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (5, 5), strides: (2, 2), prepadding: (2, 2), postpadding: (2, 2), dilations: (1, 1), numOutputs: 128
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 128, 70, 230)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 43) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 76 for ONNX tensor: 76
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [76 -> (80, 128, 70, 230)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 76
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [76 -> (80, 128, 70, 230)], [conv2.1.weight -> (128)], [conv2.1.bias -> (128)], [conv2.1.running_mean -> (128)], [conv2.1.running_var -> (128)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 44) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 77 for ONNX tensor: 77
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [77 -> (80, 128, 70, 230)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 77
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [77 -> (80, 128, 70, 230)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 45) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 78 for ONNX tensor: 78
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [78 -> (80, 128, 70, 230)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 78
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [78 -> (80, 128, 70, 230)], [conv3.0.weight -> (256, 128, 5, 5)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 128, 70, 230)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (5, 5), strides: (2, 2), prepadding: (2, 2), postpadding: (2, 2), dilations: (1, 1), numOutputs: 256
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 256, 35, 115)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 46) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 79 for ONNX tensor: 79
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [79 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 79
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [79 -> (80, 256, 35, 115)], [conv3.1.weight -> (256)], [conv3.1.bias -> (256)], [conv3.1.running_mean -> (256)], [conv3.1.running_var -> (256)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 47) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 80 for ONNX tensor: 80
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [80 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 80
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [80 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 48) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 81 for ONNX tensor: 81
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [81 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 81
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3_1.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [81 -> (80, 256, 35, 115)], [conv3_1.0.weight -> (256, 256, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 256, 35, 115)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 256
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 256, 35, 115)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 49) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 82 for ONNX tensor: 82
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [82 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 82
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3_1.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3_1.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3_1.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv3_1.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [82 -> (80, 256, 35, 115)], [conv3_1.1.weight -> (256)], [conv3_1.1.bias -> (256)], [conv3_1.1.running_mean -> (256)], [conv3_1.1.running_var -> (256)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 50) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 83 for ONNX tensor: 83
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [83 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 83
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [83 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 51) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 84 for ONNX tensor: 84
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [84 -> (80, 256, 35, 115)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 84
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [84 -> (80, 256, 35, 115)], [conv4.0.weight -> (512, 256, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 256, 35, 115)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (2, 2), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 512, 18, 58)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 52) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 85 for ONNX tensor: 85
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [85 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 85
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [85 -> (80, 512, 18, 58)], [conv4.1.weight -> (512)], [conv4.1.bias -> (512)], [conv4.1.running_mean -> (512)], [conv4.1.running_var -> (512)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 53) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 86 for ONNX tensor: 86
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [86 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 86
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [86 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 54) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 87 for ONNX tensor: 87
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [87 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 87
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4_1.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [87 -> (80, 512, 18, 58)], [conv4_1.0.weight -> (512, 512, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 512, 18, 58)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 512, 18, 58)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 55) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 88 for ONNX tensor: 88
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [88 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 88
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4_1.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4_1.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4_1.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv4_1.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [88 -> (80, 512, 18, 58)], [conv4_1.1.weight -> (512)], [conv4_1.1.bias -> (512)], [conv4_1.1.running_mean -> (512)], [conv4_1.1.running_var -> (512)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 56) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 89 for ONNX tensor: 89
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [89 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 89
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [89 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 57) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 90 for ONNX tensor: 90
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [90 -> (80, 512, 18, 58)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 90
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [90 -> (80, 512, 18, 58)], [conv5.0.weight -> (512, 512, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 512, 18, 58)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (2, 2), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 512, 9, 29)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 58) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 91 for ONNX tensor: 91
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [91 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 91
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [91 -> (80, 512, 9, 29)], [conv5.1.weight -> (512)], [conv5.1.bias -> (512)], [conv5.1.running_mean -> (512)], [conv5.1.running_var -> (512)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 59) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 92 for ONNX tensor: 92
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [92 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 92
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [92 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 60) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 93 for ONNX tensor: 93
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [93 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 93
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5_1.0.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [93 -> (80, 512, 9, 29)], [conv5_1.0.weight -> (512, 512, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 512, 9, 29)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 512, 9, 29)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 61) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 94 for ONNX tensor: 94
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [94 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [BatchNormalization]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 94
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5_1.1.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5_1.1.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5_1.1.running_mean
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv5_1.1.running_var
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [BatchNormalization] inputs: [94 -> (80, 512, 9, 29)], [conv5_1.1.weight -> (512)], [conv5_1.1.bias -> (512)], [conv5_1.1.running_mean -> (512)], [conv5_1.1.running_var -> (512)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 62) [Scale] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 95 for ONNX tensor: 95
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [BatchNormalization] outputs: [95 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LeakyRelu]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 95
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LeakyRelu] inputs: [95 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 63) [Activation] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 96 for ONNX tensor: 96
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LeakyRelu] outputs: [96 -> (80, 512, 9, 29)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Conv]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 96
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv6.weight
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Conv] inputs: [96 -> (80, 512, 9, 29)], [conv6.weight -> (1024, 512, 3, 3)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:446: Convolution input dimensions: (80, 512, 9, 29)
[06/17/2020-16:34:47] [V] [TRT] Bias weights are not set yet. Bias weights can be set using setInput(2, bias_tensor) API call.
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:528: Using kernel: (3, 3), strides: (2, 2), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 1024
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:529: Convolution output dimensions: (80, 1024, 5, 15)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 64) [Convolution] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 97 for ONNX tensor: 97
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Conv] outputs: [97 -> (80, 1024, 5, 15)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [98 -> (3)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Reshape]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 97
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 98
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Reshape] inputs: [97 -> (80, 1024, 5, 15)], [98 -> (3)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 65) [Shuffle] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 99 for ONNX tensor: 99
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Reshape] outputs: [99 -> (16, 5, 76800)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Shape]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 99
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Shape] inputs: [99 -> (16, 5, 76800)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 66) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 100 for ONNX tensor: 100
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Shape] outputs: [100 -> (3)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [101 -> ()], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Gather]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 100
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 101
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Gather] inputs: [100 -> (3)], [101 -> ()], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:978: Using Gather axis: 0
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 67) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 102 for ONNX tensor: 102
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Gather] outputs: [102 -> ()], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Unsqueeze]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 102
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Unsqueeze] inputs: [102 -> ()], 
[06/17/2020-16:34:47] [V] [TRT] onnx2trt_utils.cpp:1574: Original shape: (), unsqueezing to: (1,)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 69) [Shuffle] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 106 for ONNX tensor: 106
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Unsqueeze] outputs: [106 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Concat]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 252
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 106
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 253
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Concat] inputs: [252 -> (1)], [106 -> (1)], [253 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 70) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 108 for ONNX tensor: 108
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Concat] outputs: [108 -> (3)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [ConstantOfShape]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 108
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [ConstantOfShape] inputs: [108 -> (3)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 73) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 109 for ONNX tensor: 109
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [ConstantOfShape] outputs: [109 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Transpose]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 99
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Transpose] inputs: [99 -> (16, 5, 76800)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 77) [Shuffle] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 110 for ONNX tensor: 110
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Transpose] outputs: [110 -> (5, 16, 76800)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [168 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [169 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [170 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 109
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 169
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 170
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 168
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [109 -> (-1, -1, -1)], [169 -> (1)], [170 -> (1)], [168 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 78) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 171 for ONNX tensor: 171
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [171 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [172 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [173 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [174 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 109
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 173
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 174
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 172
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [109 -> (-1, -1, -1)], [173 -> (1)], [174 -> (1)], [172 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 97) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 175 for ONNX tensor: 175
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [175 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LSTM]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 110
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 271
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 272
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 273
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 171
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 175
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LSTM] inputs: [110 -> (5, 16, 76800)], [271 -> (1, 4000, 76800)], [272 -> (1, 4000, 1000)], [273 -> (1, 8000)], [optional input, not set], [171 -> (-1, -1, -1)], [175 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1790: Bias shape is: (1, 8000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1794: Reshaping bias to: (1, 2, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1799: After reduction, bias shape is: (1, 1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1808: numDirectionsTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1812: hiddenSizeTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1814: batchSizeTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1821: Gate output rank (equal to initial hidden/cell state rank): (3)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1835: Initial hidden state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1838: Initial cell state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1840: Entering Loop
[06/17/2020-16:34:47] [V] [TRT] onnx2trt_utils.cpp:1574: Original shape: (16, 76800), unsqueezing to: (1, 16, 76800)
[06/17/2020-16:34:47] [V] [TRT] RNNHelpers.cpp:42: Input shape: (1, 16, 76800)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1852: Hidden state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1856: Cell state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1864: X(t) * W^T -> (1, 16, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1870: H(t-1) * R^T -> (-1, -1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1877: intermediate(t) -> (-1, 16, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1948: c(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1957: C(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1980: H(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 116) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 176 for ONNX tensor: 176
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 177 for ONNX tensor: 177
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 178 for ONNX tensor: 178
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LSTM] outputs: [176 -> (-1, -1, -1, -1)], [177 -> (-1, -1, -1)], [178 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Squeeze]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 176
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Squeeze] inputs: [176 -> (-1, -1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] onnx2trt_utils.cpp:1430: Original shape: (_, _, _, _), squeezing to: (_, _, _)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 158) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 179 for ONNX tensor: 179
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Squeeze] outputs: [179 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [236 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [237 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [238 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 109
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 237
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 238
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 236
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [109 -> (-1, -1, -1)], [237 -> (1)], [238 -> (1)], [236 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 162) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 239 for ONNX tensor: 239
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [239 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [240 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [241 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Constant]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Constant] inputs: 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Constant] outputs: [242 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Slice]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 109
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 241
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 242
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 240
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Slice] inputs: [109 -> (-1, -1, -1)], [241 -> (1)], [242 -> (1)], [240 -> (1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 181) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 243 for ONNX tensor: 243
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Slice] outputs: [243 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [LSTM]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 179
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 291
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 292
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 293
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 239
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 243
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [LSTM] inputs: [179 -> (-1, -1, -1)], [291 -> (1, 4000, 1000)], [292 -> (1, 4000, 1000)], [293 -> (1, 8000)], [optional input, not set], [239 -> (-1, -1, -1)], [243 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1790: Bias shape is: (1, 8000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1794: Reshaping bias to: (1, 2, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1799: After reduction, bias shape is: (1, 1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1808: numDirectionsTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1812: hiddenSizeTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1814: batchSizeTensor shape: (1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1821: Gate output rank (equal to initial hidden/cell state rank): (3)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1835: Initial hidden state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1838: Initial cell state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1840: Entering Loop
[06/17/2020-16:34:47] [V] [TRT] onnx2trt_utils.cpp:1574: Original shape: (_, _), unsqueezing to: (_, _, _)
[06/17/2020-16:34:47] [V] [TRT] RNNHelpers.cpp:42: Input shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1852: Hidden state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1856: Cell state shape: (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1864: X(t) * W^T -> (-1, -1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1870: H(t-1) * R^T -> (-1, -1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1877: intermediate(t) -> (-1, -1, 4000)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1948: c(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1957: C(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] builtin_op_importers.cpp:1980: H(t) -> (-1, -1, -1)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 200) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 244 for ONNX tensor: 244
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 245 for ONNX tensor: 245
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 246 for ONNX tensor: 246
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [LSTM] outputs: [244 -> (-1, -1, -1, -1)], [245 -> (-1, -1, -1)], [246 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Squeeze]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 244
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Squeeze] inputs: [244 -> (-1, -1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] onnx2trt_utils.cpp:1430: Original shape: (_, _, _, _), squeezing to: (_, _, _)
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 253) [Shape] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 247 for ONNX tensor: 247
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Squeeze] outputs: [247 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Transpose]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 247
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Transpose] inputs: [247 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 257) [Shuffle] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 248 for ONNX tensor: 248
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Transpose] outputs: [248 -> (-1, -1, -1)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [MatMul]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 248
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 294
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [MatMul] inputs: [248 -> (-1, -1, -1)], [294 -> (1000, 6)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 258) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: 250 for ONNX tensor: 250
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [MatMul] outputs: [250 -> (-1, -1, 6)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:103: Parsing node:  [Add]
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: 250
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:119: Searching for input: linear.bias
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:125:  [Add] inputs: [250 -> (-1, -1, 6)], [linear.bias -> (6)], 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 261) [Constant] for ONNX node: 
[06/17/2020-16:34:47] [V] [TRT] ImporterContext.hpp:97: Registering tensor: output_1 for ONNX tensor: output
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:182:  [Add] outputs: [output -> (-1, -1, 6)], 
[06/17/2020-16:34:47] [V] [TRT] ModelImporter.cpp:494: Marking output_1 as output: output
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
[06/17/2020-16:34:47] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
 ----- Parsing of ONNX model deepvo.onnx is Done ---- 
[06/17/2020-16:34:47] [V] [TRT] Applying generic optimizations to the graph for inference.
[06/17/2020-16:34:47] [V] [TRT] Original: 105 layers
[06/17/2020-16:34:47] [V] [TRT] After dead-layer removal: 105 layers
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 73) [Constant] with (Unnamed Layer* 74) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 118) [Constant] with (Unnamed Layer* 119) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 202) [Constant] with (Unnamed Layer* 203) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 65) [Shuffle] with (Unnamed Layer* 77) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 256) [Shuffle] with (Unnamed Layer* 257) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 258) [Constant] with (Unnamed Layer* 259) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] Fusing (Unnamed Layer* 261) [Constant] with (Unnamed Layer* 262) [Shuffle]
[06/17/2020-16:34:47] [V] [TRT] After Myelin optimization: 34 layers
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 40) [Convolution] with scale (Unnamed Layer* 41) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 43) [Convolution] with scale (Unnamed Layer* 44) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 46) [Convolution] with scale (Unnamed Layer* 47) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 49) [Convolution] with scale (Unnamed Layer* 50) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 52) [Convolution] with scale (Unnamed Layer* 53) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 55) [Convolution] with scale (Unnamed Layer* 56) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 58) [Convolution] with scale (Unnamed Layer* 59) [Scale]
[06/17/2020-16:34:47] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 61) [Convolution] with scale (Unnamed Layer* 62) [Scale]
[06/17/2020-16:34:47] [V] [TRT] After scale fusion: 26 layers
[06/17/2020-16:34:47] [V] [TRT] After vertical fusions: 26 layers
[06/17/2020-16:34:47] [V] [TRT] After final dead-layer removal: 26 layers
[06/17/2020-16:34:47] [V] [TRT] After tensor merging: 26 layers
[06/17/2020-16:34:47] [V] [TRT] Eliminating concatenation (Unnamed Layer* 38) [Concatenation]
[06/17/2020-16:34:47] [V] [TRT] Retargeting 64 to 70
[06/17/2020-16:34:47] [V] [TRT] Retargeting 69 to 70
[06/17/2020-16:34:47] [V] [TRT] After concat removal: 25 layers
[06/17/2020-16:34:47] [V] [TRT] Graph construction and optimization completed in 0.155402 seconds.
[06/17/2020-16:34:47] [I] [TRT] 
[06/17/2020-16:34:47] [I] [TRT] --------------- Layers running on DLA: 
[06/17/2020-16:34:47] [I] [TRT] 
[06/17/2020-16:34:47] [I] [TRT] --------------- Layers running on GPU: 
[06/17/2020-16:34:47] [I] [TRT] (Unnamed Layer* 118) [Constant] + (Unnamed Layer* 119) [Shuffle], (Unnamed Layer* 202) [Constant] + (Unnamed Layer* 203) [Shuffle], (Unnamed Layer* 18) [Slice], (Unnamed Layer* 37) [Slice], (Unnamed Layer* 39) [Shuffle], (Unnamed Layer* 40) [Convolution], (Unnamed Layer* 204) [Reduce], (Unnamed Layer* 120) [Reduce], (Unnamed Layer* 42) [Activation], (Unnamed Layer* 43) [Convolution], (Unnamed Layer* 45) [Activation], (Unnamed Layer* 46) [Convolution], (Unnamed Layer* 48) [Activation], (Unnamed Layer* 49) [Convolution], (Unnamed Layer* 51) [Activation], (Unnamed Layer* 52) [Convolution], (Unnamed Layer* 54) [Activation], (Unnamed Layer* 55) [Convolution], (Unnamed Layer* 57) [Activation], (Unnamed Layer* 58) [Convolution], (Unnamed Layer* 60) [Activation], (Unnamed Layer* 61) [Convolution], (Unnamed Layer* 63) [Activation], (Unnamed Layer* 64) [Convolution], {(Unnamed Layer* 116) [Constant],(Unnamed Layer* 117) [Constant],(Unnamed Layer* 125) [Constant],(Unnamed Layer* 65) [Shuffle] + (Unnamed Layer* 77) [Shuffle],(Unnamed Layer* 73) [Constant] + (Unnamed Layer* 74) [Shuffle],(Unnamed Layer* 75) [Slice],(Unnamed Layer* 96) [Slice],(Unnamed Layer* 115) [Slice],(Unnamed Layer* 126) [TripLimit],(Unnamed Layer* 127) [Iterator],(Unnamed Layer* 128) [Shuffle],(Unnamed Layer* 131) [Matrix Multiply],(Unnamed Layer* 129) [Recurrence],(Unnamed Layer* 130) [Recurrence],(Unnamed Layer* 132) [Matrix Multiply],(Unnamed Layer* 133) [ElementWise],(Unnamed Layer* 134) [ElementWise],(Unnamed Layer* 135) [Slice],(Unnamed Layer* 138) [Slice],(Unnamed Layer* 141) [Slice],(Unnamed Layer* 147) [Slice],(Unnamed Layer* 137) [Activation],(Unnamed Layer* 140) [Activation],(Unnamed Layer* 143) [Activation],(Unnamed Layer* 149) [Activation],(Unnamed Layer* 144) [ElementWise],(Unnamed Layer* 145) [ElementWise],(Unnamed Layer* 146) [ElementWise],(Unnamed Layer* 150) [Activation],(Unnamed Layer* 151) [ElementWise],(Unnamed Layer* 155) [LoopOutput],(Unnamed Layer* 213) [Gather]_output[Constant],(Unnamed Layer* 200) [Constant],(Unnamed Layer* 201) [Constant],(Unnamed Layer* 180) [Slice],(Unnamed Layer* 199) [Slice],(Unnamed Layer* 161) [Shuffle],(Unnamed Layer* 222) [Recurrence],(Unnamed Layer* 223) [Recurrence],(Unnamed Layer* 225) [Matrix Multiply],(Unnamed Layer* 215) [Iterator],(Unnamed Layer* 214) [TripLimit],(Unnamed Layer* 221) [Shuffle],(Unnamed Layer* 224) [Matrix Multiply],(Unnamed Layer* 226) [ElementWise],(Unnamed Layer* 227) [ElementWise],(Unnamed Layer* 228) [Slice],(Unnamed Layer* 231) [Slice],(Unnamed Layer* 234) [Slice],(Unnamed Layer* 240) [Slice],(Unnamed Layer* 230) [Activation],(Unnamed Layer* 233) [Activation],(Unnamed Layer* 236) [Activation],(Unnamed Layer* 242) [Activation],(Unnamed Layer* 237) [ElementWise],(Unnamed Layer* 238) [ElementWise],(Unnamed Layer* 239) [ElementWise],(Unnamed Layer* 243) [Activation],(Unnamed Layer* 244) [ElementWise],(Unnamed Layer* 250) [LoopOutput],(Unnamed Layer* 256) [Shuffle] + (Unnamed Layer* 257) [Shuffle],(Unnamed Layer* 258) [Constant] + (Unnamed Layer* 259) [Shuffle],(Unnamed Layer* 260) [Matrix Multiply],(Unnamed Layer* 261) [Constant] + (Unnamed Layer* 262) [Shuffle],(Unnamed Layer* 263) [ElementWise]}, 
Killed

I’m using TensorRT 7.1.0 with Python 3 and Cuda 10.2 on a Jetson Xavier NX.

AakankshaS · June 17, 2020, 6:02pm

Hi,
Moving this to Jetson Xavier NX forum so that Jetson team can take a look.
Thanks!

AastaLLL · June 19, 2020, 3:07am

Hi,

1.3Gb is a pretty complicated model.
Have you checked how many memory it takes when runs with pyTorch on a desktop environment.
Since XavierNX is an embedded system, it will be better to check if the resource is enough first.

Thanks.

lvkaleo · June 19, 2020, 10:19am

Hi,

thanks for you reply.
Running the model with Onnxruntime took ~ 3.5 GB RAM.

However, I tried using a smaller model with reduced capacity (using smaller image sizes and a lower number of channels in the hidden state of the LSTM which resulted in an onnx model size of ~200 MB) and the TensorRT engine was created succesfully on Xavier NX.

AastaLLL · June 22, 2020, 6:10am

Hi,

3.5GB memory is large.
Would you mind to run it with TensorRT and check system status at the same time.

$ sudo tegrastats

Thanks.

lvkaleo · June 22, 2020, 8:56am

Hi,

the full model doesn’t fit into memory.

With the smaller version of the model, memory usage of the TensorRT engine creation process goes up to ~ 5800/7772 MB. Here’s a small snippet of the output of tegrastats:

RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [76%@1267,5%@1267,off,off,off,off] EMC_FREQ 23%@1600 GR3D_FREQ 93%@803 APE 75 MTS fg 0% bg 0% AO@38C GPU@39C PMIC@100C AUX@39C CPU@39C thermal@39C VDD_IN 5593/4829 VDD_CPU_GPU_CV 1796/1525 VDD_SOC 1549/1216
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [77%@1190,5%@1190,off,off,off,off] EMC_FREQ 22%@1600 GR3D_FREQ 94%@803 APE 75 MTS fg 0% bg 0% AO@38C GPU@39C PMIC@100C AUX@39C CPU@39C thermal@38.85C VDD_IN 5757/4832 VDD_CPU_GPU_CV 1837/1526 VDD_SOC 1549/1217
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [76%@1190,5%@1190,off,off,off,off] EMC_FREQ 20%@1600 GR3D_FREQ 94%@803 APE 75 MTS fg 0% bg 0% AO@38.5C GPU@39.5C PMIC@100C AUX@39C CPU@39C thermal@38.85C VDD_IN 6083/4836 VDD_CPU_GPU_CV 1959/1528 VDD_SOC 1630/1219
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [76%@1190,5%@1190,off,off,off,off] EMC_FREQ 29%@1600 GR3D_FREQ 94%@803 APE 75 MTS fg 0% bg 0% AO@38.5C GPU@40C PMIC@100C AUX@39.5C CPU@39.5C thermal@38.85C VDD_IN 7145/4843 VDD_CPU_GPU_CV 2572/1531 VDD_SOC 1834/1221
RAM 5795/7772MB (lfb 171x4MB) SWAP 4/3886MB (cached 0MB) CPU [78%@1497,6%@1497,off,off,off,off] EMC_FREQ 33%@1600 GR3D_FREQ 44%@803 APE 75 MTS fg 0% bg 0% AO@38.5C GPU@39.5C PMIC@100C AUX@39C CPU@39.5C thermal@39.05C VDD_IN 7023/4850 VDD_CPU_GPU_CV 2572/1534 VDD_SOC 1793/1223
RAM 5849/7772MB (lfb 171x4MB) SWAP 4/3886MB (cached 0MB) CPU [82%@1497,5%@1497,off,off,off,off] EMC_FREQ 26%@1600 GR3D_FREQ 0%@803 APE 75 MTS fg 0% bg 0% AO@38.5C GPU@39C PMIC@100C AUX@39.5C CPU@39C thermal@39.5C VDD_IN 6165/4854 VDD_CPU_GPU_CV 2000/1536 VDD_SOC 1630/1224
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [53%@1190,22%@1190,off,off,off,off] EMC_FREQ 32%@1600 GR3D_FREQ 96%@803 APE 75 MTS fg 0% bg 0% AO@38.5C GPU@39.5C PMIC@100C AUX@39.5C CPU@39.5C thermal@39.5C VDD_IN 6737/4860 VDD_CPU_GPU_CV 2449/1539 VDD_SOC 1633/1225
RAM 5811/7772MB (lfb 171x4MB) SWAP 4/3886MB (cached 0MB) CPU [47%@1497,17%@1497,off,off,off,off] EMC_FREQ 31%@1600 GR3D_FREQ 34%@803 APE 75 MTS fg 0% bg 0% AO@39C GPU@39.5C PMIC@100C AUX@39.5C CPU@39.5C thermal@39.65C VDD_IN 6696/4866 VDD_CPU_GPU_CV 2531/1542 VDD_SOC 1551/1226
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [38%@1497,41%@1495,off,off,off,off] EMC_FREQ 45%@1600 GR3D_FREQ 99%@803 APE 75 MTS fg 1% bg 2% AO@39C GPU@40C PMIC@100C AUX@39.5C CPU@40C thermal@39.65C VDD_IN 7962/4875 VDD_CPU_GPU_CV 3144/1547 VDD_SOC 1837/1228
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [6%@1190,13%@1190,off,off,off,off] EMC_FREQ 107%@1600 GR3D_FREQ 94%@803 APE 75 MTS fg 0% bg 3% AO@39.5C GPU@40C PMIC@100C AUX@40C CPU@40C thermal@39.8C VDD_IN 8084/4885 VDD_CPU_GPU_CV 2899/1551 VDD_SOC 2038/1231
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [17%@1190,27%@1190,off,off,off,off] EMC_FREQ 99%@1600 GR3D_FREQ 29%@803 APE 75 MTS fg 0% bg 5% AO@39.5C GPU@39.5C PMIC@100C AUX@40C CPU@40C thermal@40C VDD_IN 6288/4890 VDD_CPU_GPU_CV 2123/1553 VDD_SOC 1671/1232
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [12%@1190,21%@1190,off,off,off,off] EMC_FREQ 115%@1600 GR3D_FREQ 99%@803 APE 75 MTS fg 0% bg 6% AO@39C GPU@40.5C PMIC@100C AUX@40C CPU@40C thermal@40C VDD_IN 7186/4897 VDD_CPU_GPU_CV 2490/1556 VDD_SOC 1875/1234
RAM 5777/7772MB (lfb 173x4MB) SWAP 4/3886MB (cached 0MB) CPU [13%@1190,15%@1190,off,off,off,off] EMC_FREQ 117%@1600 GR3D_FREQ 99%@803 APE 75 MTS fg 0% bg 4% AO@39C GPU@40C PMIC@100C AUX@40.5C CPU@40C thermal@40.35C VDD_IN 6573/4902 VDD_CPU_GPU_CV 2245/1558 VDD_SOC 1712/1235

Performing Inference with TensorRT with this model consumes about 950 MB memory.

Do you think if I create the TensorRT engine for the large model on the server it was trained on, I would be able to run it on Jetson Xavier NX?

Thanks.

AastaLLL · July 16, 2020, 2:09am

Hi,

Unfortunately, no.

Please noticed that TensorRT engine is not portable.
So you will need to create the engine on the XavierNX directly.

Thanks.