[ArcFace] [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I am trying to run arcface in tensorrt.

First download model from onnx model zoo

When I convert onnx to tensorrt engine,using:

def build_engine_onnx(model_file):
    with trt.Builder(TRT_LOGGER) as builder:
        with builder.create_network() as network:
            with trt.OnnxParser(network, TRT_LOGGER) as parser:
                builder.max_workspace_size = common.GiB(1)
                builder.max_batch_size = batch_size
                # Load the Onnx model and parse it in order to populate the TensorRT network.
                with open(model_file, 'rb') as model:
                    parser.parse(model.read())
                return builder.build_cuda_engine(network)

[TensorRT] ERROR: Network must have at least one output

./trtexec --onnx=resnet100.onnx

[07/07/2020-14:18:39] [I] === Model Options ===
[07/07/2020-14:18:39] [I] Format: ONNX
[07/07/2020-14:18:39] [I] Model: resnet100.onnx
[07/07/2020-14:18:39] [I] Output:
[07/07/2020-14:18:39] [I] === Build Options ===
[07/07/2020-14:18:39] [I] Max batch: 1
[07/07/2020-14:18:39] [I] Workspace: 16 MB
[07/07/2020-14:18:39] [I] minTiming: 1
[07/07/2020-14:18:39] [I] avgTiming: 8
[07/07/2020-14:18:39] [I] Precision: FP32
[07/07/2020-14:18:39] [I] Calibration: 
[07/07/2020-14:18:39] [I] Safe mode: Disabled
[07/07/2020-14:18:39] [I] Save engine: 
[07/07/2020-14:18:39] [I] Load engine: 
[07/07/2020-14:18:39] [I] Builder Cache: Enabled
[07/07/2020-14:18:39] [I] NVTX verbosity: 0
[07/07/2020-14:18:39] [I] Inputs format: fp32:CHW
[07/07/2020-14:18:39] [I] Outputs format: fp32:CHW
[07/07/2020-14:18:39] [I] Input build shapes: model
[07/07/2020-14:18:39] [I] Input calibration shapes: model
[07/07/2020-14:18:39] [I] === System Options ===
[07/07/2020-14:18:39] [I] Device: 0
[07/07/2020-14:18:39] [I] DLACore: 
[07/07/2020-14:18:39] [I] Plugins:
[07/07/2020-14:18:39] [I] === Inference Options ===
[07/07/2020-14:18:39] [I] Batch: 1
[07/07/2020-14:18:39] [I] Input inference shapes: model
[07/07/2020-14:18:39] [I] Iterations: 10
[07/07/2020-14:18:39] [I] Duration: 3s (+ 200ms warm up)
[07/07/2020-14:18:39] [I] Sleep time: 0ms
[07/07/2020-14:18:39] [I] Streams: 1
[07/07/2020-14:18:39] [I] ExposeDMA: Disabled
[07/07/2020-14:18:39] [I] Spin-wait: Disabled
[07/07/2020-14:18:39] [I] Multithreading: Disabled
[07/07/2020-14:18:39] [I] CUDA Graph: Disabled
[07/07/2020-14:18:39] [I] Skip inference: Disabled
[07/07/2020-14:18:39] [I] Inputs:
[07/07/2020-14:18:39] [I] === Reporting Options ===
[07/07/2020-14:18:39] [I] Verbose: Disabled
[07/07/2020-14:18:39] [I] Averages: 10 inferences
[07/07/2020-14:18:39] [I] Percentile: 99
[07/07/2020-14:18:39] [I] Dump output: Disabled
[07/07/2020-14:18:39] [I] Profile: Disabled
[07/07/2020-14:18:39] [I] Export timing to JSON file: 
[07/07/2020-14:18:39] [I] Export output to JSON file: 
[07/07/2020-14:18:39] [I] Export profile to JSON file: 
[07/07/2020-14:18:39] [I] 
----------------------------------------------------------------
Input filename:   resnet100.onnx
ONNX IR version:  0.0.3
Opset version:    8
Producer name:    
Producer version: 
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[07/07/2020-14:18:43] [E] [TRT] (Unnamed Layer* 12) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
ERROR: onnx2trt_utils.cpp:1364 In function scaleHelper:
[8] Assertion failed: dims.nbDims == 4 || dims.nbDims == 5
[07/07/2020-14:18:43] [E] Failed to parse onnx file
[07/07/2020-14:18:43] [E] Parsing model failed
[07/07/2020-14:18:43] [E] Engine creation failed
[07/07/2020-14:18:43] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=resnet100.onnx

Any suggestions for arcface in python tensorrt?

Environment

Jetson nano, latest Jetpack4.4DP

2

Hi @S1NH,

This usually means that the TensorRT ONNX parser failed to parse your model.
You can check the below code snippet for reference
https://github.com/jkjung-avt/tensorrt_demos/blob/master/yolov3_onnx/onnx_to_tensorrt.py#L59

This issue comes usually when there is some wrong with the model, however I was not able to reproduce it on TRT version 7.1 using the following command
trtexec --onnx=resnet100.onnx --verbose --explicitBatch --shapes=input:1x3x112x112
Also check the below link for your reference.
https://github.com/onnx/onnx/blob/master/docs/Operators.md#PRelu

Thanks!