TRT 7 - NHWC support?

Hi,

Historically, TensorRT has only supported NCHW I/O data and models/engines.

However now I see here that a NHWC input is being used.

There’s no documentation about this in the TensorRT Developer Guide.

So: can you clarify:

  • Is NHWC is supported?
  • Is it only for I/O or also internally inside the model?
  • What are the implications of using NHWC over NCHW? (e.g. will it be slower?)
  • What format conversions does TRT internally?

Thanks!

Hi,

NHWC is supported in TRT.
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-700/tensorrt-developer-guide/index.html#data-format-desc
Using NHWC will have some benefit according to device you use. In general:

  • with tensor cores, NHWC is (generally) preferred.
  • for non-tensor-core, NCHW is preferred.

Thanks

Nice, thanks!

Is there any reference I can read about “NHWC is preferred for Tensor Cores”?

Hi,

It’s just a generally preferred format. But it may vary based on the scenario and device you use.
I don’t have any public reference link currently available for reference.

Thanks