Does Tile/SLICE operator support BOOL type?

AI & Data Science Deep Learning (Training & Inference) TensorRT
#1

Description

I know that Tile op in ONNX converts into Slice op in TensorRT.
When exporting ONNX model to TRT engine, it seems we don’t support Tile operator that has BOOL type input and output. Instead of giving a readable error, it throws an exception.

onnx-tensorrt/operators.md at master · onnx/onnx-tensorrt · GitHub Here they support BOOL for Tile

Support Matrix :: NVIDIA Deep Learning TensorRT Documentation But here ISlice layer doesn’t support BOOL.

Environment

TensorRT Version: 8.0
GPU Type: V100\T4
Nvidia Driver Version:460
CUDA Version: 11
CUDNN Version: 8.2
Operating System + Version:RHEL
Python Version (if applicable): 3.8

Relevant Files

None

Steps To Reproduce

#2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html#onnx-export

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

#4

As long as the onnx model has a Tile op that has BOOL input output. The error is

Error[2]: [optimizer.cpp::getFormatRequirements::3600] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed.no supported formats)

Same error for trtexec command

#5

Hi,

Could you please share us issue repro ONNX model to try from our end for better debugging.

Thank you.

#6

tile.onnx (294 Bytes)

Hi, here is a tiny file that contains about only the tile op. To reproduce we have to disable Myelin. My original model is too big and inconvenient to upload. I found in my origin model, The BOOL type Tile is not clustered into Myelin Node and thows error. When trying to reproduce using the minimum subgraph, I found there are no errors when all nodes can be clustered into Myelin Node.
I’m using TRT8.0, and the error should be:
[TensorRT] ERROR: 2: [optimizer.cpp::getFormatRequirements::3600] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed.no supported formats)
thanks very much.

#7

Actually I have an opensource version of TensorRT8.0, now I have modified slice.cu and sliceBuilder.cpp to get the BOOL type slice supported.
Out of Myelin graph the types seem to be not fully supported, for example Less Op seems to have similiar problem.