TensoRT convewrsion in bfloat16

This doc state that orin NX support bfloat16

However it is not available when converting a ONNX model to tensorRT.

/usr/src/tensorrt/bin/trtexec --help
  --noTF32                    Disable tf32 precision (default is to enable tf32, in addition to fp32)
  --fp16                      Enable fp16 precision, in addition to fp32 (default = disabled)
  --int8                      Enable int8 precision, in addition to fp32 (default = disabled)
  --best                      Enable all precisions to achieve the best performance (default = disabled)

Same here, only classic FP16 and no bfloat16.

How to enable TensoRT convewrsion in bfloat16 ?


Glad to know you found the solution, thanks for your sharing!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.