Description
When I use torch.onnx.export with do_constant_folding=True, the model converts to onnx, TensorRT without error, but has accuracy issues, so I want to try to convert the model with do_constant_folding=False, but then converting the model with trtexec returns this error:
D:\pyth\pytracking-master\pytracking>trtexec --onnx=tomp101_head_latest4.onnx --saveEngine=latest4_fp32.engine --useCudaGraph --explicitBatch --verbose > log_fp32_cftrue.txt
[01/14/2024-19:41:47] [W] --explicitBatch flag has been deprecated and has no effect!
[01/14/2024-19:41:47] [W] Explicit batch dim is automatically enabled if input model is ONNX or if dynamic shapes are provided when the engine is built.
[01/14/2024-19:41:54] [W] [TRT] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/14/2024-19:41:54] [W] [TRT] onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped
[01/14/2024-19:41:54] [E] [TRT] ModelImporter.cpp:771: While parsing node number 130 [Slice -> "/pos_encoding/Slice_output_0"]:
[01/14/2024-19:41:54] [E] [TRT] ModelImporter.cpp:772: --- Begin node ---
[01/14/2024-19:41:54] [E] [TRT] ModelImporter.cpp:773: input: "/pos_encoding/CumSum_output_0"
input: "/pos_encoding/Unsqueeze_1_output_0"
input: "/pos_encoding/Unsqueeze_2_output_0"
input: "/pos_encoding/Unsqueeze_output_0"
input: "/pos_encoding/Unsqueeze_3_output_0"
output: "/pos_encoding/Slice_output_0"
name: "/pos_encoding/Slice"
op_type: "Slice"
[01/14/2024-19:41:54] [E] [TRT] ModelImporter.cpp:774: --- End node ---
[01/14/2024-19:41:54] [E] [TRT] ModelImporter.cpp:777: ERROR: builtin_op_importers.cpp:4493 In function importSlice:
[8] Assertion failed: (axes.allValuesKnown()) && "This version of TensorRT does not support dynamic axes."
[01/14/2024-19:41:54] [E] Failed to parse onnx file
[01/14/2024-19:41:54] [E] Parsing model failed
[01/14/2024-19:41:54] [E] Failed to create engine from model or file.
[01/14/2024-19:41:54] [E] Engine set up failed
the only code that uses /pos_encoding/:
class PositionEmbeddingSine(nn.Module):
"""
This is a more standard version of the position embedding, very similar to the one
used by the Attention is all you need paper, generalized to work on images.
"""
def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None,
sine_type='lin_sine', avoid_aliazing=False, max_spatial_resolution=None):
super().__init__()
self.num_pos_feats = num_pos_feats
self.temperature = temperature
self.normalize = normalize
self.sine = NerfPositionalEncoding(num_pos_feats//2, sine_type, avoid_aliazing, max_spatial_resolution)
@torch.no_grad()
def forward(self, mask):
assert mask is not None
not_mask = ~mask
y_embed = not_mask.cumsum(1, dtype=torch.float32)
x_embed = not_mask.cumsum(2, dtype=torch.float32)
eps = 1e-6
y_embed = (y_embed-0.5) / (y_embed[:, -1:, :] + eps)
x_embed = (x_embed-0.5) / (x_embed[:, :, -1:] + eps)
pos = torch.stack([x_embed, y_embed], dim=-1)
return self.sine(pos).permute(0, 3, 1, 2)
pos_encoding = PositionEmbeddingSine(num_pos_feats=self.transformer.d_model//2, sine_type='lin_sine',
avoid_aliazing=True, max_spatial_resolution=feature_sz)
mask = torch.zeros((nframes * nseq, h, w), dtype=torch.bool, device=feat.device)
pos = self.pos_encoding(mask)
CAN ANYBODY with experience explain what is wrong here and why TensorRT doesn’t like it?
the code used for pytorch to onnx conversion:
input_names = ['sample_x', 'train_samples', 'target_labels', 'train_ltrb']
output_names = ['target_scores', 'bbreg_test_feat_enc', 'bbreg_weights']
torch.onnx.export(model,
model_inputs,
"tomp101_head_latest4.onnx",
verbose=False,
export_params=True,
do_constant_folding=False,
opset_version=16,
input_names=input_names,
output_names=output_names,
dynamic_axes={'sample_x':{0:'batch_size'},
'train_samples':{0:'batch_size'},
'target_labels':{0:'batch_size'},
'train_ltrb':{0:'batch_size'},
'target_scores':{0:'batch_size'},
'bbreg_test_feat_enc':{0:'batch_size'},
'bbreg_weights':{0:'batch_size'}})
If any extra info is required please let me know!!!
Thank you
Environment
TensorRT Version: 8.6
GPU Type: GTX 1660 Ti
Nvidia Driver Version: 546.01
CUDA Version: 12.1
CUDNN Version: 8.9.7
Operating System + Version: Windows 10
Python Version (if applicable): 3.10.13
PyTorch Version: 2.1.2+cu121
Baremetal or Container (if container which image + tag): No environment, running straight on Windows 10
Relevant Files
the onnx model: https://drive.google.com/file/d/1oDQD-aSuitAGAPkyXtnFUUUnvCoSSC9f/view?usp=sharing
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered