How to create a plugin support input format::kHWC8?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I want to create a plugin that support input DataType::kHALF and TensorFormat::kHWC8.

Here’s my function:

bool MyPlugin::supportsFormatCombination(
int32_t pos, const PluginTensorDesc *inOut, int32_t nbInputs, int32_t nbOutputs)
{
// input and output must have the same dataType
return inOut[pos].type == DataType::kHALF &&
inOut[pos].format == TensorFormat::kLINEAR;
}

It works fine with kLINEAR.
But when use kHWC8 instead of kLINEAR, I got error:

[04/18/2021-17:44:56] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[04/18/2021-17:46:21] [E] [TRT] ../rtSafe/cuda/cudaReformatRunner.cpp (251) - Assertion Error in combineDHtoH: 0 (nbSpatialDims == 3 && nbDims >= 5)
[04/18/2021-17:46:21] [E] Engine creation failed
[04/18/2021-17:46:21] [E] Engine set up failed.

2

Hi,
Please refer to below links related custom plugin implementation and sample:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMnistCoordConvAC

Thanks!

3

I have tried the examples in TensorRT/plugin at master · NVIDIA/TensorRT · GitHub, but all of them only support TensorFormat::kLINEAR and this leads to extra layers in network. Is there any way to write plugin support kHWC8?
Thanks!

4

Hi @807907950,

We request you to share issue reproducible scripts/model for better assistance.

Thank you.

5

Hi, I met the same problem. Do you solve it?

Reformat error for plugin with input format kHWC
7

Hi，I upgrade my TensorRT to 7.2.2.3 and solved.

8

Hi, I met the same problem.