root@77e4471577ce:/workspace/tensorrt/bin# ./trtexec --onnx="./whereTest.onnx" --int8 --inputIOFormats=int8:chw --verbose &&&& RUNNING TensorRT.trtexec # ./trtexec --onnx=/home/kristof/optimum/cloudsorter_6d/notebooks/export_models/whereTest.onnx --int8 --inputIOFormats=int8:chw --verbose [03/11/2021-00:18:54] [I] === Model Options === [03/11/2021-00:18:54] [I] Format: ONNX [03/11/2021-00:18:54] [I] Model: /home/kristof/optimum/cloudsorter_6d/notebooks/export_models/whereTest.onnx [03/11/2021-00:18:54] [I] Output: [03/11/2021-00:18:54] [I] === Build Options === [03/11/2021-00:18:54] [I] Max batch: 1 [03/11/2021-00:18:54] [I] Workspace: 16 MB [03/11/2021-00:18:54] [I] minTiming: 1 [03/11/2021-00:18:54] [I] avgTiming: 8 [03/11/2021-00:18:54] [I] Precision: FP32+INT8 [03/11/2021-00:18:54] [I] Calibration: Dynamic [03/11/2021-00:18:54] [I] Safe mode: Disabled [03/11/2021-00:18:54] [I] Save engine: [03/11/2021-00:18:54] [I] Load engine: [03/11/2021-00:18:54] [I] Builder Cache: Enabled [03/11/2021-00:18:54] [I] NVTX verbosity: 0 [03/11/2021-00:18:54] [I] Input: int8:chw [03/11/2021-00:18:54] [I] Outputs format: fp32:CHW [03/11/2021-00:18:54] [I] Input build shapes: model [03/11/2021-00:18:54] [I] Input calibration shapes: model [03/11/2021-00:18:54] [I] === System Options === [03/11/2021-00:18:54] [I] Device: 0 [03/11/2021-00:18:54] [I] DLACore: [03/11/2021-00:18:54] [I] Plugins: [03/11/2021-00:18:54] [I] === Inference Options === [03/11/2021-00:18:54] [I] Batch: 1 [03/11/2021-00:18:54] [I] Input inference shapes: model [03/11/2021-00:18:54] [I] Iterations: 10 [03/11/2021-00:18:54] [I] Duration: 3s (+ 200ms warm up) [03/11/2021-00:18:54] [I] Sleep time: 0ms [03/11/2021-00:18:54] [I] Streams: 1 [03/11/2021-00:18:54] [I] ExposeDMA: Disabled [03/11/2021-00:18:54] [I] Spin-wait: Disabled [03/11/2021-00:18:54] [I] Multithreading: Disabled [03/11/2021-00:18:54] [I] CUDA Graph: Disabled [03/11/2021-00:18:54] [I] Skip inference: Disabled [03/11/2021-00:18:54] [I] Inputs: [03/11/2021-00:18:54] [I] === Reporting Options === [03/11/2021-00:18:54] [I] Verbose: Enabled [03/11/2021-00:18:54] [I] Averages: 10 inferences [03/11/2021-00:18:54] [I] Percentile: 99 [03/11/2021-00:18:54] [I] Dump output: Disabled [03/11/2021-00:18:54] [I] Profile: Disabled [03/11/2021-00:18:54] [I] Export timing to JSON file: [03/11/2021-00:18:54] [I] Export output to JSON file: [03/11/2021-00:18:54] [I] Export profile to JSON file: [03/11/2021-00:18:54] [I] [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::CropAndResize version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::Proposal version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::Region_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1 [03/11/2021-00:18:54] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1 ---------------------------------------------------------------- Input filename: /home/kristof/optimum/cloudsorter_6d/notebooks/export_models/whereTest.onnx ONNX IR version: 0.0.6 Opset version: 11 Producer name: tf2onnx Producer version: 1.8.3 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::Proposal version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:206: Adding network input: input:0 with dtype: float32, dimensions: (4096, 6) [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: input:0 for ONNX tensor: input:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: StatefulPartitionedCall/SelectV2/e:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: StatefulPartitionedCall/Greater/y:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/random_uniform/RandomUniform [RandomUniform] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/random_uniform/RandomUniform [RandomUniform] inputs: [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/random_uniform/RandomUniform for ONNX node: StatefulPartitionedCall/random_uniform/RandomUniform [03/11/2021-00:19:06] [W] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:2355: TensorRT currently ignores the "seed" field in RandomUniform op. Random seeds will be used. [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/random_uniform/RandomUniform:0 for ONNX tensor: StatefulPartitionedCall/random_uniform/RandomUniform:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/random_uniform/RandomUniform [RandomUniform] outputs: [StatefulPartitionedCall/random_uniform/RandomUniform:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/Greater [Greater] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/random_uniform/RandomUniform:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/Greater/y:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/Greater [Greater] inputs: [StatefulPartitionedCall/random_uniform/RandomUniform:0 -> (4096, 6)], [StatefulPartitionedCall/Greater/y:0 -> ()], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/Greater for ONNX node: StatefulPartitionedCall/Greater [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/Greater:0 for ONNX tensor: StatefulPartitionedCall/Greater:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/Greater [Greater] outputs: [StatefulPartitionedCall/Greater:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2__7 [Not] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/Greater:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2__7 [Not] inputs: [StatefulPartitionedCall/Greater:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2__7 for ONNX node: StatefulPartitionedCall/SelectV2__7 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/SelectV2__7:0 for ONNX tensor: StatefulPartitionedCall/SelectV2__7:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2__7 [Not] outputs: [StatefulPartitionedCall/SelectV2__7:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2__8 [Cast] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2__7:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2__8 [Cast] inputs: [StatefulPartitionedCall/SelectV2__7:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:320: Casting to type: float32 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2__8 for ONNX node: StatefulPartitionedCall/SelectV2__8 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/SelectV2__8:0 for ONNX tensor: StatefulPartitionedCall/SelectV2__8:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2__8 [Cast] outputs: [StatefulPartitionedCall/SelectV2__8:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2__10 [Mul] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2/e:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2__8:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2__10 [Mul] inputs: [StatefulPartitionedCall/SelectV2/e:0 -> ()], [StatefulPartitionedCall/SelectV2__8:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2__10 for ONNX node: StatefulPartitionedCall/SelectV2__10 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/SelectV2__10:0 for ONNX tensor: StatefulPartitionedCall/SelectV2__10:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2__10 [Mul] outputs: [StatefulPartitionedCall/SelectV2__10:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2__6 [Cast] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/Greater:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2__6 [Cast] inputs: [StatefulPartitionedCall/Greater:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:320: Casting to type: float32 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2__6 for ONNX node: StatefulPartitionedCall/SelectV2__6 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/SelectV2__6:0 for ONNX tensor: StatefulPartitionedCall/SelectV2__6:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2__6 [Cast] outputs: [StatefulPartitionedCall/SelectV2__6:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2__9 [Mul] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2__6:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: input:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2__9 [Mul] inputs: [StatefulPartitionedCall/SelectV2__6:0 -> (4096, 6)], [input:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2__9 for ONNX node: StatefulPartitionedCall/SelectV2__9 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: StatefulPartitionedCall/SelectV2__9:0 for ONNX tensor: StatefulPartitionedCall/SelectV2__9:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2__9 [Mul] outputs: [StatefulPartitionedCall/SelectV2__9:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: StatefulPartitionedCall/SelectV2 [Add] [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2__9:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: StatefulPartitionedCall/SelectV2__10:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: StatefulPartitionedCall/SelectV2 [Add] inputs: [StatefulPartitionedCall/SelectV2__9:0 -> (4096, 6)], [StatefulPartitionedCall/SelectV2__10:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:144: Registering layer: StatefulPartitionedCall/SelectV2 for ONNX node: StatefulPartitionedCall/SelectV2 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: Identity:0_0 for ONNX tensor: Identity:0 [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: StatefulPartitionedCall/SelectV2 [Add] outputs: [Identity:0 -> (4096, 6)], [03/11/2021-00:19:06] [V] [TRT] [TRT]/home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:535: Marking Identity:0_0 as output: Identity:0 ----- Parsing of ONNX model /home/kristof/optimum/cloudsorter_6d/notebooks/export_models/whereTest.onnx is Done ---- [03/11/2021-00:19:06] [I] FP32 and INT8 precisions have been specified - more performance might be enabled by additionally specifying --fp16 or --best [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/random_uniform/RandomUniform:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for (Unnamed Layer* 1) [Constant]_output to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for (Unnamed Layer* 2) [Shuffle]_output to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/Greater:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/SelectV2__7:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/SelectV2__8:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for (Unnamed Layer* 6) [Constant]_output to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for (Unnamed Layer* 7) [Shuffle]_output to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/SelectV2__10:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/SelectV2__6:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for input:0 to [-2,2] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for StatefulPartitionedCall/SelectV2__9:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Setting dynamic range for Identity:0 to [-4,4] [03/11/2021-00:19:06] [V] [TRT] Applying generic optimizations to the graph for inference. [03/11/2021-00:19:06] [V] [TRT] Original: 12 layers [03/11/2021-00:19:06] [V] [TRT] After dead-layer removal: 12 layers [03/11/2021-00:19:06] [V] [TRT] Fusing (Unnamed Layer* 1) [Constant] with (Unnamed Layer* 2) [Shuffle] [03/11/2021-00:19:06] [V] [TRT] Fusing (Unnamed Layer* 6) [Constant] with (Unnamed Layer* 7) [Shuffle] [03/11/2021-00:19:06] [V] [TRT] After Myelin optimization: 2 layers [03/11/2021-00:19:06] [V] [TRT] After scale fusion: 2 layers [03/11/2021-00:19:06] [V] [TRT] After vertical fusions: 2 layers [03/11/2021-00:19:06] [V] [TRT] After final dead-layer removal: 2 layers [03/11/2021-00:19:06] [V] [TRT] After tensor merging: 2 layers [03/11/2021-00:19:06] [V] [TRT] After concat removal: 2 layers [03/11/2021-00:19:06] [V] [TRT] Graph construction and optimization completed in 0.00106508 seconds. [03/11/2021-00:19:06] [V] [TRT] Constructing optimization profile number 0 [1/1]. [03/11/2021-00:19:06] [V] [TRT] *************** Autotuning format combination: -> Float(1,6) *************** [03/11/2021-00:19:06] [V] [TRT] --------------- Timing Runner: StatefulPartitionedCall/random_uniform/RandomUniform (Fill) [03/11/2021-00:19:06] [V] [TRT] Tactic: 0 is the only option, timing skipped [03/11/2021-00:19:06] [V] [TRT] Fastest Tactic: 0 Time: 0 [03/11/2021-00:19:06] [V] [TRT] Builder timing cache: created 0 entries, 0 hit(s) [03/11/2021-00:19:06] [E] [TRT] ../builder/cudnnBuilder2.cpp (1917) - Assertion Error in makePaddedScale: 0 (regionRanges != nullptr) [03/11/2021-00:19:06] [E] Engine creation failed [03/11/2021-00:19:06] [E] Engine set up failed &&&& FAILED TensorRT.trtexec # ./trtexec --onnx=./whereTest.onnx --int8 --inputIOFormats=int8:chw --verbose