Description
A clear and concise description of the bug or issue.
Environment
TensorRT Version: 8.2.1.8
GPU Type: jetson xarive nx 16g
Nvidia Driver Version: jetpack4.6.1
CUDA Version: 10.2
CUDNN Version: I dont kown,its installed by jetpack4.6.1.
Operating System + Version: ubuntu 18.04LTS
Python Version (if applicable): 3.6.9
I use a .py to realize onnx to trt, error:
trt version 8.2.1.8
[03/18/2022-16:54:16] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[03/18/2022-16:54:16] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[03/18/2022-16:54:18] [TRT] [E] 2: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{1794, 6, 16}, 12653},)
Traceback (most recent call last):
File "tools/export_trt.py", line 77, in <module>
f.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'
.py:
import tensorrt as trt
import sys
import argparse
"""
takes in onnx model
converts to tensorrt
"""
if __name__ == '__main__':
desc = 'compile Onnx model to TensorRT'
parser = argparse.ArgumentParser(description=desc)
parser.add_argument('--model', help='onnx file')
parser.add_argument('--out', type=str, default='', help='name of trt output file')
parser.add_argument('--fp', type=int, default=16, help='floating point precision. 16 or 32')
parser.add_argument('--batch', type=int, default=1)
parser.add_argument('--verbose', action='store_true')
parser.add_argument('--jetson_nano', action='store_true')
opt = parser.parse_args()
batch_size = opt.batch
model = opt.model
fp = opt.fp
output = opt.out if opt.out else opt.model.replace('.onnx', '.trt')
assert fp in (16, 32)
if opt.jetson_nano:
workspace = 1 << 28
else:
workspace = 4 * 1 << 30
logger = trt.Logger(trt.Logger.WARNING)
if opt.verbose:
logger.min_severity = trt.Logger.VERBOSE
print('trt version', trt.__version__)
assert trt.__version__[0] >= '7'
EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
with trt.Builder(logger) as builder, builder.create_network(EXPLICIT_BATCH) as network, trt.OnnxParser(network, logger) as parser:
if trt.__version__[0] == '7':
builder.max_workspace_size = workspace
builder.max_batch_size = batch_size
if fp == 16:
builder.fp16_mode = True
with open(model, 'rb') as f:
if not parser.parse(f.read()):
for error in range(parser.num_errors):
print('ERROR', parser.get_error(error))
# if your onnx has a dynamic input...
# network.get_input(0).shape = [1, 3, 352, 608]
engine = builder.build_cuda_engine(network)
with open(output, 'wb') as f:
f.write(engine.serialize())
print('Done')
else:
# https://github.com/NVIDIA-AI-IOT/torch2trt/issues/557
# https://github.com/NVIDIA-AI-IOT/torch2trt/commit/8f742904d603fcde4fe521baa31bdc18002c23cb#diff-f682ce583d8646e112002fb3631f7d205b63aae7b0ca673b020fed7244d4ed38
config = builder.create_builder_config()
config.max_workspace_size = workspace
if fp == 16:
config.set_flag(trt.BuilderFlag.FP16)
builder.max_batch_size = batch_size
with open(model, 'rb') as f:
if not parser.parse(f.read()):
for error in range(parser.num_errors):
print('ERROR', parser.get_error(error))
engine = builder.build_engine(network, config)
with open(output, 'wb') as f:
f.write(engine.serialize())
print('Done')
it can work in jetson nano with jetpack4.5, should I install jetpack 4.5 to nx?