Dynamic input and convolution

Hello,

I was trying to use the new dynamic input shapes introduced by TensorRT 6. However during the engine creation a error occurred. My logger says “ERROR: C:\source\rtSafe\cuda\caskConvolutionRunner.cpp (62) - Cask Error in nvinfer1::rt::task::CaskConvolutionRunner::createConvolution: 3 (isConsistent)”. I think this error is caused by a combination of using kSame padding with a stride size of more than one. Below I have a minimal code example which leads to the error.

#include <NvInfer.h>
#include <iostream>
#include <vector>

using namespace std;
using namespace nvinfer1;

class StdOutLogger : public ILogger
{
public:
  StdOutLogger(ILogger::Severity severity)
    : mReportedSeverity(severity)
  {}

  void log(ILogger::Severity severity, char const* message) override
  {
    if (severity > mReportedSeverity)
      return;

    switch (severity)
    {
    case Severity::kINTERNAL_ERROR: std::cerr << "INTERNAL_ERROR: "; break;
    case Severity::kERROR: std::cerr << "ERROR: "; break;
    case Severity::kWARNING: std::cerr << "WARNING: "; break;
    case Severity::kINFO: std::cerr << "INFO: "; break;
    default: std::cerr << "UNKNOWN: "; break;
    }

    std::cerr << message << std::endl;
  }

private:
  ILogger::Severity mReportedSeverity;
};

Weights getKernelWeights()
{
  Weights weights;
  weights.count = 18;
  weights.type = DataType::kFLOAT;
  vector<float> values{ 1.f, 2.f, 3.f, 4.f, 5.f, 6.f, 7.f, 8.f, 9.f, 10.f, 11.f, 12.f, 13.f, 14.f, 15.f, 16.f, 17.f, 18.f };
  weights.values = values.data();
  return weights;
}

Weights getBiasWeights()
{
  Weights weights;
  weights.count = 2;
  weights.type = DataType::kFLOAT;
  vector<float> values{ -1.f, -2.f };
  weights.values = values.data();
  return weights;
}

int main()
{
  StdOutLogger logger(ILogger::Severity::kINFO);
  auto builder = createInferBuilder(logger);
  auto network = builder->createNetworkV2(1U << static_cast<size_t>(NetworkDefinitionCreationFlag::kEXPLICIT_BATCH));

  auto convKernelWeights = getKernelWeights();
  auto convBiasWeights = getBiasWeights();

  auto input = network->addInput("input", DataType::kFLOAT, Dims4{ 1, 1, -1, -1 });
  auto conv = network->addConvolution(*input, 2, DimsHW{ 3, 3 }, convKernelWeights, convBiasWeights);

  // here the error occurs, without padding everything works fine
  conv->setPaddingMode(PaddingMode::kSAME_LOWER);

  // however if I change the strides to (1, 1) it works with the padding
  conv->setStride(DimsHW{ 2, 2 });

  network->markOutput(*conv->getOutput(0));

  auto profile = builder->createOptimizationProfile();
  profile->setDimensions("input", OptProfileSelector::kMIN, Dims4{ 1, 1, 64, 64 });
  profile->setDimensions("input", OptProfileSelector::kOPT, Dims4{ 1, 1, 128, 128 });
  profile->setDimensions("input", OptProfileSelector::kMAX, Dims4{ 1, 1, 256, 256 });

  auto config = builder->createBuilderConfig();
  config->addOptimizationProfile(profile);

  auto engine = builder->buildEngineWithConfig(*network, *config);

  engine->destroy();
  config->destroy();
  builder->destroy();

  return 0;
}

So again the error is ERROR: C:\source\rtSafe\cuda\caskConvolutionRunner.cpp (62) - Cask Error in nvinfer1::rt::task::CaskConvolutionRunner::createConvolution: 3 (isConsistent)

Thank you in advance.

Hi Xalanot,

I was able to repro your issue and have escalated to the engineering team for more details.

Thanks,
NVIDIA Enterprise Support

Hi Xalanot,

Per engineering team, the root cause of this issue comes from CUDNN not supporting asymmetric padding (related link: https://github.com/JuliaGPU/CuArrays.jl/issues/356)

There was a bug in how they handled this CUDNN issue, it should be fixed in the next release. Thanks for pointing this out!

What version was this resolved in?

Hi @curiousguy,

I believe TensorRT 7.0. If this is not the case, please open a separate issue with your environment/version info and a script to reproduce it per the issue template.