TensoRT convewrsion in bfloat16

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

This doc state that orin NX support bfloat16

image
image1178×139 20.5 KB

However it is not available when converting a ONNX model to tensorRT.

/usr/src/tensorrt/bin/trtexec --help
  --noTF32                    Disable tf32 precision (default is to enable tf32, in addition to fp32)
  --fp16                      Enable fp16 precision, in addition to fp32 (default = disabled)
  --int8                      Enable int8 precision, in addition to fp32 (default = disabled)
  --best                      Enable all precisions to achieve the best performance (default = disabled)

Same here, only classic FP16 and no bfloat16.
https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/FoundationalTypes/DataType.html

How to enable TensoRT convewrsion in bfloat16 ?

2

solved:

3

Glad to know you found the solution, thanks for your sharing!

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.