NHWC inputs/outputs

[solaris.insight]

Hello, could you please tell me, does TensorRT support NHWC inputs without internal forced transpose of inputs? If not, why?

Internally all convolutions are NHWC anyway for Tensor Cores so why is there a NCHW input requirement?
Can we have a simple flag that disables the internal transpose of inputs and outputs?

[spolisetty]

TensorRT supports both NHCW and NCHW. It automatically taken care by TRT,
TRT internally tries all kinds of tensor layouts.

[solaris.insight]

TensorRT supports NHWC inputs?

[spolisetty]

Please refer,

TRT 7 - NHWC support?

Hi, NHWC is supported in TRT. https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-700/tensorrt-developer-guide/index.html#data-format-desc Using NHWC will have some benefit according to device you use. In general: with tensor cores, NHWC is (generally) preferred. for non-tensor-core, NCHW is preferred. Thanks

[solaris.insight]

Thank you. How do i specify that the input to the network is in NHWC format?

[spolisetty]

You can use “setAllowedFormats” API for input tensor, see on NvInfer.h
https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_tensor.html