Hello,
I have a keras 3d convolutional model that I would like to optimize on Tensor RT. I have successfully converted the .h5 into a .pb, and the .pb into a onnx file. I am trying to optimize this onnx model with tensor rt with this script:
However this returns a plugin error for max pooling 3d. I am not sure why I am receiving this error because I have tensor rt 7 and I thought it provides support for 3d convolution and pooling, How do I solve this?
Hi,
There are some limitation in the 3D pooling.
Would you mind to check if your model is supported by both TensorRT API and onnx2trt first?
TensorRT : Support Matrix :: NVIDIA Deep Learning TensorRT Documentation
ONNX2TRT : https://github.com/onnx/onnx-tensorrt/blob/master/operators.md
Thanks.
From what I’ve read, it shows support for 2d and 3d for pooling. However, I am unsure on max pooling because Onnx2TRT simply says yes for max pooling. Release Notes :: NVIDIA Deep Learning TensorRT Documentation shows support for pooling.
Here is my onnx file in case you want to try it out:
https://drive.google.com/file/d/1Doj9sZWI7KrBT2_-aTQoNhxZx6Zxvdb3/view?usp=sharing
Hi,
Sorry for the late update.
It looks like ONNX2TRT only support MaxPool and no MaxPool3D operation support.
// Finally add the scale layer.
auto layer = ctx->network()->addScale(
tensor, nvinfer1::ScaleMode::kCHANNEL, shiftWeights, scaleWeights, nvinfer1::Weights{});
ctx->registerLayer(layer, getNodeName(node));
RETURN_FIRST_OUTPUT(layer);
}
DEFINE_BUILTIN_OP_IMPORTER(InstanceNormalization)
{
// Scales and biases must be initializers
ASSERT(inputs.at(1).is_weights() && "The scale tensor is required to be an initializer.", ErrorCode::kUNSUPPORTED_NODE);
ASSERT(inputs.at(2).is_weights() && "The bias tensor is required to be an initializer.", ErrorCode::kUNSUPPORTED_NODE);
nvinfer1::ITensor* tensorPtr = &convertToTensor(inputs.at(0), ctx);
int nbDims = tensorPtr->getDimensions().nbDims;
ASSERT(nbDims >= 3 && nbDims <= 5 && "TensorRT only supports InstanceNormalization on 3D, 4D, or 5D tensors!",
ErrorCode::kUNSUPPORTED_NODE);
const bool needToExpandDims = (nbDims == 3);
if (needToExpandDims)
{
// Expand spatial dims from 1D to 2D
Here are two possible workarounds for your reference.
Add the MaxPool3D
support into ONNX2TRT .
Update the operation into MaxPool when generating the onnx model from your .pb file.
Thanks.