Hi,
Historically, TensorRT has only supported NCHW I/O data and models/engines.
However now I see here that a NHWC input is being used.
There’s no documentation about this in the TensorRT Developer Guide.
So: can you clarify:
- Is NHWC is supported?
- Is it only for I/O or also internally inside the model?
- What are the implications of using NHWC over NCHW? (e.g. will it be slower?)
- What format conversions does TRT internally?
Thanks!
Hi,
NHWC is supported in TRT.
Using NHWC will have some benefit according to device you use. In general:
- with tensor cores, NHWC is (generally) preferred.
- for non-tensor-core, NCHW is preferred.
Thanks
Nice, thanks!
Is there any reference I can read about “NHWC is preferred for Tensor Cores”?
Hi,
It’s just a generally preferred format. But it may vary based on the scenario and device you use.
I don’t have any public reference link currently available for reference.
Thanks