Myelin error while converting to tensor engine

aaryan · August 11, 2020, 4:47am

Description

while converting an onnx model to tensorrt engine, I’m getting following myelin error. Can you guide what can be the issue?
(I’m using ngc tensorrt container in docker)

Error:

[08/10/2020-09:55:04] [V] [TRT] Builder timing cache: created 31 entries, 4 hit(s)
[08/10/2020-09:55:04] [E] [TRT] …/builder/myelin/codeGenerator.cpp (396) - Myelin Error in compileGraph: 64 (myelinProgramAnalysisError : Mistmatched number of elements in op lambda_1_1/Reshape_11_reshape _ reshape_(Unnamed Layer_ 355) [Constant](Unnamed Layer 356) [Constant](Unnamed Layer 374) [Constant](Unnamed Layer 349) [Constant] _ (Unnamed Layer_ 350) [Shuffle](Unnamed Layer 346) [Constant] _ (Unnamed Layer_ 347) [Shuffle]lambda_1_1/sub_8_lambda_1_1/sub_9_lambda_1_1/mul_11_lambda_1_1/ExpandDims_4_lambda_1_1/mul_15_lambda_1_1/sub_6_lambda_1_1/sub_7_lambda_1_1/mul_10_lambda_1_1/ExpandDims_3_lambda_1_1/mul_14_lambda_1_1/sub_5_lambda_1_1/sub_4_lambda_1_1/mul_9_lambda_1_1/ExpandDims_2_lambda_1_1/mul_13_lambda_1_1/sub_2_lambda_1_1/sub_3_lambda_1_1/mul_8_lambda_1_1/ExpandDims_1_lambda_1_1/mul_12_lambda_1_1/AddN_sum_lambda_1_1/AddN_sum_1_lambda_1_1/AddN_sum_2_lambda_1_1/Reshape_11_reshape _ reshape_fc_9_Add2_Relu_Transpose4(Unnamed Layer_ 366) [Constant] _ (Unnamed Layer_ 367) [Shuffle]lstm_10(Unnamed Layer_ 370) [Constant] _ (Unnamed Layer_ 371) [Shuffle]lstm_10_1(Unnamed Layer_ 412) [Constant](Unnamed Layer 413) [Constant](Unnamed Layer 431) [Constant]Transpose6(Unnamed Layer_ 423) [Constant] _ (Unnamed Layer_ 424) [Shuffle]lstm_10_back(Unnamed Layer_ 427) [Constant] _ (Unnamed Layer_ 428) [Shuffle]lstm_10_back_1(Unnamed Layer_ 432) [TripLimit](Unnamed Layer 433) [Iterator]lstm_10_back_2(Unnamed Layer_ 441) [Recurrence](Unnamed Layer 439) [Shuffle](Unnamed Layer 443) [Matrix Multiply](Unnamed Layer 442) [Matrix Multiply](Unnamed Layer 444) [ElementWise](Unnamed Layer 445) [ElementWise](Unnamed Layer 446) [Slice](Unnamed Layer 449) [Slice](Unnamed Layer 452) [Slice](Unnamed Layer 458) [Slice](Unnamed Layer 448) [Activation](Unnamed Layer 451) [Activation](Unnamed Layer 454) [Activation](Unnamed Layer 460) [Activation](Unnamed Layer 456) [ElementWise](Unnamed Layer 455) [ElementWise](Unnamed Layer 457) [ElementWise](Unnamed Layer 461) [Activation](Unnamed Layer 462) [ElementWise](Unnamed Layer 466) [LoopOutput](Unnamed Layer 375) [TripLimit](Unnamed Layer 376) [Iterator]lstm_10_2(Unnamed Layer_ 384) [Recurrence](Unnamed Layer 382) [Shuffle](Unnamed Layer 386) [Matrix Multiply](Unnamed Layer 385) [Matrix Multiply](Unnamed Layer 387) [ElementWise](Unnamed Layer 388) [ElementWise](Unnamed Layer 389) [Slice](Unnamed Layer 392) [Slice](Unnamed Layer 395) [Slice](Unnamed Layer 401) [Slice](Unnamed Layer 391) [Activation](Unnamed Layer 394) [Activation](Unnamed Layer 397) [Activation](Unnamed Layer 403) [Activation](Unnamed Layer 399) [ElementWise](Unnamed Layer 398) [ElementWise](Unnamed Layer 400) [ElementWise](Unnamed Layer 404) [Activation](Unnamed Layer 405) [ElementWise](Unnamed Layer 409) [LoopOutput](Unnamed Layer 478) [Constant](Unnamed Layer 479) [Constant](Unnamed Layer 489) [Constant] _ (Unnamed Layer_ 490) [Shuffle]lstm_11(Unnamed Layer_ 493) [Constant] _ (Unnamed Layer_ 494) [Shuffle]lstm_11_1_Squeeze2_Squeeze3_Add1(Unnamed Layer_ 539) [Constant](Unnamed Layer 540) [Constant](Unnamed Layer 550) [Constant] _ (Unnamed Layer_ 551) [Shuffle]lstm_11_back(Unnamed Layer_ 554) [Constant] _ (Unnamed Layer_ 555) [Shuffle]lstm_11_back_1(Unnamed Layer_ 562) [Iterator](Unnamed Layer 561) [TripLimit]lstm_11_back_2(Unnamed Layer_ 570) [Recurrence](Unnamed Layer 568) [Shuffle](Unnamed Layer 572) [Matrix Multiply](Unnamed Layer 571) [Matrix Multiply](Unnamed Layer 573) [ElementWise](Unnamed Layer 574) [ElementWise](Unnamed Layer 575) [Slice](Unnamed Layer 578) [Slice](Unnamed Layer 581) [Slice](Unnamed Layer 587) [Slice](Unnamed Layer 577) [Activation](Unnamed Layer 580) [Activation](Unnamed Layer 583) [Activation](Unnamed Layer 589) [Activation](Unnamed Layer 585) [ElementWise](Unnamed Layer 584) [ElementWise](Unnamed Layer 586) [ElementWise](Unnamed Layer 590) [Activation](Unnamed Layer 591) [ElementWise](Unnamed Layer 597) [LoopOutput]Squeeze1(Unnamed Layer_ 501) [Iterator](Unnamed Layer 500) [TripLimit]lstm_11_2(Unnamed Layer_ 509) [Recurrence](Unnamed Layer 507) [Shuffle](Unnamed Layer 511) [Matrix Multiply](Unnamed Layer 510) [Matrix Multiply](Unnamed Layer 512) [ElementWise](Unnamed Layer 513) [ElementWise](Unnamed Layer 514) [Slice](Unnamed Layer 517) [Slice](Unnamed Layer 520) [Slice](Unnamed Layer 526) [Slice](Unnamed Layer 516) [Activation](Unnamed Layer 519) [Activation](Unnamed Layer 522) [Activation](Unnamed Layer 528) [Activation](Unnamed Layer 524) [ElementWise](Unnamed Layer 523) [ElementWise](Unnamed Layer 525) [ElementWise](Unnamed Layer 529) [Activation](Unnamed Layer 530) [ElementWise](Unnamed Layer 536) [LoopOutput]Squeeze_concatenate_TransposeFanIn_succ_0(Unnamed Layer_ 610) [Constant] _ (Unnamed Layer_ 611) [Shuffle]fc_12(Unnamed Layer_ 613) [Constant] _ (Unnamed Layer_ 614) [Shuffle]Add(Unnamed Layer_ 624) [Shuffle]Softmax(Unnamed Layer_ 626) [Shuffle]fc_12_1/Identity_0_cropping__reshape input tensor lambda_1_1/AddN_0’(Unnamed Layer_ 355) [Constant](Unnamed Layer 356) [Constant](Unnamed Layer 374) [Constant](Unnamed Layer 349) [Constant] _ (Unnamed Layer_ 350) [Shuffle](Unnamed Layer 346) [Constant] _ (Unnamed Layer_ 347) [Shuffle]lambda_1_1/sub_8_lambda_1_1/sub_9_lambda_1_1/mul_11_lambda_1_1/ExpandDims_4_lambda_1_1/mul_15_lambda_1_1/sub_6_lambda_1_1/sub_7_lambda_1_1/mul_10_lambda_1_1/ExpandDims_3_lambda_1_1/mul_14_lambda_1_1/sub_5_lambda_1_1/sub_4_lambda_1_1/mul_9_lambda_1_1/ExpandDims_2_lambda_1_1/mul_13_lambda_1_1/sub_2_lambda_1_1/sub_3_lambda_1_1/mul_8_lambda_1_1/ExpandDims_1_lambda_1_1/mul_12_lambda_1_1/AddN_sum_lambda_1_1/AddN_sum_1_lambda_1_1/AddN_sum_2_lambda_1_1/Reshape_11_reshape _ reshape_fc_9_Add2_Relu_Transpose4(Unnamed Layer_ 366) [Constant] _ (Unnamed Layer_ 367) [Shuffle]lstm_10(Unnamed Layer_ 370) [Constant] _ (Unnamed Layer_ 371) [Shuffle]lstm_10_1(Unnamed Layer_ 412) [Constant](Unnamed Layer 413) [Constant](Unnamed Layer 431) [Constant]Transpose6(Unnamed Layer_ 423) [Constant] _ (Unnamed Layer_ 424) [Shuffle]lstm_10_back(Unnamed Layer_ 427) [Constant] _ (Unnamed Layer_ 428) [Shuffle]lstm_10_back_1(Unnamed Layer_ 432) [TripLimit](Unnamed Layer 433) [Iterator]lstm_10_back_2(Unnamed Layer_ 441) [Recurrence](Unnamed Layer 439) [Shuffle](Unnamed Layer 443) [Matrix Multiply](Unnamed Layer 442) [Matrix Multiply](Unnamed Layer 444) [ElementWise](Unnamed Layer 445) [ElementWise](Unnamed Layer 446) [Slice](Unnamed Layer 449) [Slice](Unnamed Layer 452) [Slice](Unnamed Layer 458) [Slice](Unnamed Layer 448) [Activation](Unnamed Layer 451) [Activation](Unnamed Layer 454) [Activation](Unnamed Layer 460) [Activation](Unnamed Layer 456) [ElementWise](Unnamed Layer 455) [ElementWise](Unnamed Layer 457) [ElementWise](Unnamed Layer 461) [Activation](Unnamed Layer 462) [ElementWise](Unnamed Layer 466) [LoopOutput](Unnamed Layer 375) [TripLimit](Unnamed Layer 376) [Iterator]lstm_10_2(Unnamed Layer_ 384) [Recurrence](Unnamed Layer 382) [Shuffle](Unnamed Layer 386) [Matrix Multiply](Unnamed Layer 385) [Matrix Multiply](Unnamed Layer 387) [ElementWise](Unnamed Layer 388) [ElementWise](Unnamed Layer 389) [Slice](Unnamed Layer 392) [Slice](Unnamed Layer 395) [Slice](Unnamed Layer 401) [Slice](Unnamed Layer 391) [Activation](Unnamed Layer 394) [Activation](Unnamed Layer 397) [Activation](Unnamed Layer 403) [Activation](Unnamed Layer 399) [ElementWise](Unnamed Layer 398) [ElementWise](Unnamed Layer 400) [ElementWise](Unnamed Layer 404) [Activation](Unnamed Layer 405) [ElementWise](Unnamed Layer 409) [LoopOutput](Unnamed Layer 478) [Constant](Unnamed Layer 479) [Constant](Unnamed Layer 489) [Constant] _ (Unnamed Layer_ 490) [Shuffle]lstm_11(Unnamed Layer_ 493) [Constant] _ (Unnamed Layer_ 494) [Shuffle]lstm_11_1_Squeeze2_Squeeze3_Add1(Unnamed Layer_ 539) [Constant](Unnamed Layer 540) [Constant](Unnamed Layer 550) [Constant] _ (Unnamed Layer_ 551) [Shuffle]lstm_11_back(Unnamed Layer_ 554) [Constant] _ (Unnamed Layer_ 555) [Shuffle]lstm_11_back_1(Unnamed Layer_ 562) [Iterator](Unnamed Layer 561) [TripLimit]lstm_11_back_2(Unnamed Layer_ 570) [Recurrence](Unnamed Layer 568) [Shuffle](Unnamed Layer 572) [Matrix Multiply](Unnamed Layer 571) [Matrix Multiply](Unnamed Layer 573) [ElementWise](Unnamed Layer 574) [ElementWise](Unnamed Layer 575) [Slice](Unnamed Layer 578) [Slice](Unnamed Layer 581) [Slice](Unnamed Layer 587) [Slice](Unnamed Layer 577) [Activation](Unnamed Layer 580) [Activation](Unnamed Layer 583) [Activation](Unnamed Layer 589) [Activation](Unnamed Layer 585) [ElementWise](Unnamed Layer 584) [ElementWise](Unnamed Layer 586) [ElementWise](Unnamed Layer 590) [Activation](Unnamed Layer 591) [ElementWise](Unnamed Layer 597) [LoopOutput]Squeeze1(Unnamed Layer_ 501) [Iterator](Unnamed Layer 500) [TripLimit]lstm_11_2(Unnamed Layer_ 509) [Recurrence](Unnamed Layer 507) [Shuffle](Unnamed Layer 511) [Matrix Multiply](Unnamed Layer 510) [Matrix Multiply](Unnamed Layer 512) [ElementWise](Unnamed Layer 513) [ElementWise](Unnamed Layer 514) [Slice](Unnamed Layer 517) [Slice](Unnamed Layer 520) [Slice](Unnamed Layer 526) [Slice](Unnamed Layer 516) [Activation](Unnamed Layer 519) [Activation](Unnamed Layer 522) [Activation](Unnamed Layer 528) [Activation](Unnamed Layer 524) [ElementWise](Unnamed Layer 523) [ElementWise](Unnamed Layer 525) [ElementWise](Unnamed Layer 529) [Activation](Unnamed Layer 530) [ElementWise](Unnamed Layer 536) [LoopOutput]Squeeze_concatenate_TransposeFanIn_succ_0(Unnamed Layer_ 610) [Constant] _ (Unnamed Layer_ 611) [Shuffle]fc_12(Unnamed Layer_ 613) [Constant] _ (Unnamed Layer_ 614) [Shuffle]Add(Unnamed Layer_ 624) [Shuffle]Softmax(Unnamed Layer_ 626) [Shuffle]fc_12_1/Identity_0_cropping shape [350,512] and specified output shape [350,50,3584]
)

[08/10/2020-11:19:44] [V] [TRT] Builder timing cache: created 31 entries, 4 hit(s)
[08/10/2020-11:19:44] [E] [TRT] …/builder/myelin/codeGenerator.cpp (396) - Myelin Error in compileGraph: 64 ()
[08/10/2020-09:55:04] [E] Engine creation failed
[08/10/2020-09:55:04] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=keras-recognize-model.onnx --explicitBatch --verbose
root@33bbf7f1ec89:/workspace#

Environment

TensorRT Version: 7.1
GPU Type: T4
CUDA Version: 11.0
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3
TensorFlow Version (if applicable): 2.2.0

Relevant Files

Here’s my model file:

Steps To Reproduce

trtexec --onnx=keras-recognize-model.onnx --verbose --explicitBatch

AakankshaS · August 11, 2020, 8:22am

Hi @aaryan,
Have you created a custom plugin here?
Can you please help me with your script to assist you better.

Thanks!

aaryan · August 11, 2020, 8:34am

@AakankshaS No, I haven’t created any custom plugin here.
Here’s Script:

pip install keras_ocr
replace recognition.py file inside keras_ocr folder with this file
run this script for converting to onnx model : keras_ocr_onnx.py - Google Drive

please check help if you can convert this model to tensorrt engine

aaryan · August 12, 2020, 9:23am

@AakankshaS just wanted to know if you got any updates on this??

AakankshaS · August 13, 2020, 7:56am

hi @aaryan,
This is a known issue, and will be fixed in upcoming releases.
Please stay tuned.
Thanks!

guneet · October 26, 2020, 8:09am

@AakankshaS I am facing a similar issue. Any updates related to this.
Can you guide on how to solve this?