Tensorrt minimum runtime on docker

1342868324 · November 15, 2023, 8:29am

Hi ,every one

I would like to know if the official provides a minimum runtime image for Tensorrt? Because the Tensorrt image on NGC is very large large image on ngc , I hope to have a lightweight runtime image. Does the official have such an image? If not, does it mean that I need to build it myself from the basic image.

Looking forward to your reply

AakankshaS · November 15, 2023, 1:05pm

Hi @1342868324 ,
If not image, you can also install trt locally using any one of the methods.

However on the lighter version, i am checking with the team.

Thanks

1342868324 · November 16, 2023, 4:48am

Thank you for your reply.

Lightweight runtime images are very helpful for deployment and management (k3s/k8s). The disk usage of images downloaded from ngc is approximately 5g-10g, and I hope it can be reduced to 0.5g-2g. I believe many companies have this issue when deploying trt images. Please check the requirements in this regard.

AakankshaS · November 16, 2023, 4:51am

Thank you for your feedback, Will pass this on to Engineering Team.

AakankshaS · November 27, 2023, 7:45am

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

github.com

onnx/onnx-tensorrt/blob/main/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.6 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:

Thanks!

1342868324 · November 27, 2023, 7:59am

Hello, thank you for your reply.

My TRT model did not have any issues with conversion and inference, I just want to know if the minimum running image of TRT can reduce size