Error[2]: [graphOptimizer.cpp::matchDetails::4222] Error Code 2: Internal Error (Assertion residual->extent == output->extent failed. )

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

the given model can be converted to tensorrt engine using tensorrt before8.4.1,
but the conversion failed in tensorrt8.5.2

Environment

TensorRT Version: 8.5.2.1
GPU Type: Orin
Nvidia Driver Version:
CUDA Version: 11.4
CUDNN Version: 8.4.1
Operating System + Version: ubuntu 20.04.4
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Steps To Reproduce

./trtexec --onnx=test.onnx --saveEngine=test.onnx.engine --minShapes=feature:1x512x4,xyz0:1x512x3,xyz1:1x128x3,xyz2:1x32x3,xyz3:1x8x3,neigh_idx0:1x512x1,neigh_idx1:1x128x8,neigh_idx2:1x32x8,neigh_idx3:1x8x8,sub_idx0:1x128x1,sub_idx1:1x32x8,sub_idx2:1x8x8,sub_idx3:1x2x8,interp_idx0:1x512x1,interp_idx1:1x128x1,interp_idx2:1x32x1,interp_idx3:1x8x1 --optShapes=feature:1x51200x4,xyz0:1x51200x3,xyz1:1x12800x3,xyz2:1x3200x3,xyz3:1x800x3,neigh_idx0:1x51200x1,neigh_idx1:1x12800x8,neigh_idx2:1x3200x8,neigh_idx3:1x800x8,sub_idx0:1x12800x1,sub_idx1:1x3200x8,sub_idx2:1x800x8,sub_idx3:1x200x8,interp_idx0:1x51200x1,interp_idx1:1x12800x1,interp_idx2:1x3200x1,interp_idx3:1x800x1 --maxShapes=feature:1x153600x4,xyz0:1x153600x3,xyz1:1x38400x3,xyz2:1x9600x3,xyz3:1x2400x3,neigh_idx0:1x153600x1,neigh_idx1:1x38400x8,neigh_idx2:1x9600x8,neigh_idx3:1x2400x8,sub_idx0:1x38400x1,sub_idx1:1x9600x8,sub_idx2:1x2400x8,sub_idx3:1x600x8,interp_idx0:1x153600x1,interp_idx1:1x38400x1,interp_idx2:1x9600x1,interp_idx3:1x2400x1 --verbose
test.onnx (363.6 KB)

2

[07/07/2023-11:55:37] [V] [TRT] After Myelin optimization: 113 layers
[07/07/2023-11:55:37] [V] [TRT] Applying ScaleNodes fusions.
[07/07/2023-11:55:37] [V] [TRT] After scale fusion: 113 layers
[07/07/2023-11:55:37] [V] [TRT] Running: SqueezePushDownFork on (Unnamed Layer* 11) [Shuffle]
[07/07/2023-11:55:37] [V] [TRT] -----------SqueezePushDown kSQUEEZE_FORK case: Conv_1 → (Unnamed Layer* 11) [Shuffle] → LeakyRelu_2
[07/07/2023-11:55:37] [V] [TRT] Running: ShuffleShuffleFusion on squeeze_after_LeakyRelu_2
[07/07/2023-11:55:37] [V] [TRT] ShuffleShuffleFusion: Fusing squeeze_after_LeakyRelu_2 with Unsqueeze_3
[07/07/2023-11:55:37] [V] [TRT] Running: ShuffleErasure on squeeze_after_LeakyRelu_2 + Unsqueeze_3
[07/07/2023-11:55:37] [V] [TRT] Removing squeeze_after_LeakyRelu_2 + Unsqueeze_3
[07/07/2023-11:55:37] [V] [TRT] Running: ReduceToPoolingFusion on ReduceMax_83
[07/07/2023-11:55:37] [V] [TRT] Swap the layer type of ReduceMax_83 from REDUCE to POOLING
[07/07/2023-11:55:37] [E] Error[2]: [graphOptimizer.cpp::matchDetails::4222] Error Code 2: Internal Error (Assertion residual->extent == output->extent failed. )
[07/07/2023-11:55:37] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[07/07/2023-11:55:37] [E] Engine could not be created from network
[07/07/2023-11:55:37] [E] Building engine failed
[07/07/2023-11:55:37] [E] Failed to create engine from model or file.
[07/07/2023-11:55:37] [E] Engine set up failed

3

Hi,

Based on the above error, it looks like the model has a residual tensor and an output tensor that have incompatible shapes. On running the Polygraphy tool constant folding, we could successfully build the engine.

/polygraphy surgeon sanitize test.onnx --fold-constants --output model_folded.onnx

[07/07/2023-05:05:18] [I]
&&&& PASSED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=model_folded.onnx --saveEngine=test.onnx.engine --minShapes=feature:1x512x4,xyz0:1x512x3,xyz1:1x128x3,xyz2:1x32x3,xyz3:1x8x3,neigh_idx0:1x512x1,neigh_idx1:1x128x8,neigh_idx2:1x32x8,neigh_idx3:1x8x8,sub_idx0:1x128x1,sub_idx1:1x32x8,sub_idx2:1x8x8,sub_idx3:1x2x8,interp_idx0:1x512x1,interp_idx1:1x128x1,interp_idx2:1x32x1,interp_idx3:1x8x1 --optShapes=feature:1x51200x4,xyz0:1x51200x3,xyz1:1x12800x3,xyz2:1x3200x3,xyz3:1x800x3,neigh_idx0:1x51200x1,neigh_idx1:1x12800x8,neigh_idx2:1x3200x8,neigh_idx3:1x800x8,sub_idx0:1x12800x1,sub_idx1:1x3200x8,sub_idx2:1x800x8,sub_idx3:1x200x8,interp_idx0:1x51200x1,interp_idx1:1x12800x1,interp_idx2:1x3200x1,interp_idx3:1x800x1 --maxShapes=feature:1x153600x4,xyz0:1x153600x3,xyz1:1x38400x3,xyz2:1x9600x3,xyz3:1x2400x3,neigh_idx0:1x153600x1,neigh_idx1:1x38400x8,neigh_idx2:1x9600x8,neigh_idx3:1x2400x8,sub_idx0:1x38400x1,sub_idx1:1x9600x8,sub_idx2:1x2400x8,sub_idx3:1x600x8,interp_idx0:1x153600x1,interp_idx1:1x38400x1,interp_idx2:1x9600x1,interp_idx3:1x2400x1 --verbose --workspace=20000

Thank you.

1 Like
4

Tests ok.
Thanks a lot.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.