[TensorRT] ERROR: 2: [pluginV2DynamicExtRunner.cpp::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed.)

Description

Hi, I have build a model that has three inputs with tensorrt python api layer by layer, and I develop a 3d grid sample plugin to build voxel. Everything runs well when building an engine, but when the engine is used fo inference there’s an error thrown. But when the plugin can be successfully implemented independently. Is anyone knows what’s the problem.

Error log

Layer(Constant): (Unnamed Layer* 329) [Constant], Tactic: 0, → (Unnamed Layer* 329) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 331) [ElementWise], (Unnamed Layer* 332) [ElementWise]), (Unnamed Layer* 333) [ElementWise]), (Unnamed Layer* 334) [ElementWise]), Tactic: 6, (Unnamed Layer* 327) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 329) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 328) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 330) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 325) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 334) [ElementWise]_output[Float(1,32,199680)]
Layer(Scale): (Unnamed Layer* 336) [Scale], Tactic: 0, (Unnamed Layer* 335) [Shuffle]_output[Float(1,64,160,624)] → (Unnamed Layer* 336) [Scale]_output[Float(1,64,160,624)]
Layer(ElementWise): (Unnamed Layer* 338) [ElementWise] + (Unnamed Layer* 339) [Activation], Tactic: 1, (Unnamed Layer* 337) [Resize]_output[Float(1,64,160,624)], (Unnamed Layer* 336) [Scale]_output[Float(1,64,160,624)] → (Unnamed Layer* 339) [Activation]_output[Float(1,64,160,624)]
Layer(Reduce): (Unnamed Layer* 512) [Reduce], Tactic: 1, (Unnamed Layer* 511) [Shuffle]_output[Float(1,32,199680)] → (Unnamed Layer* 512) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 513) [ElementWise], Tactic: 1, (Unnamed Layer* 511) [Shuffle]_output[Float(1,32,199680)], (Unnamed Layer* 512) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)]
Layer(Constant): (Unnamed Layer* 518) [Constant], Tactic: 0, → (Unnamed Layer* 518) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 516) [Constant], Tactic: 0, → (Unnamed Layer* 516) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 514) [ElementWise] + (Unnamed Layer* 515) [Reduce], Tactic: 2, (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 515) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 517) [Constant], Tactic: 0, → (Unnamed Layer* 517) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 519) [ElementWise], (Unnamed Layer* 520) [ElementWise]), (Unnamed Layer* 521) [ElementWise]), (Unnamed Layer* 522) [ElementWise]), Tactic: 6, (Unnamed Layer* 515) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 517) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 516) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 518) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 522) [ElementWise]_output[Float(1,32,199680)]
Layer(Scale): (Unnamed Layer* 524) [Scale], Tactic: 0, (Unnamed Layer* 523) [Shuffle]_output[Float(1,64,160,624)] → (Unnamed Layer* 524) [Scale]_output[Float(1,64,160,624)]
Layer(ElementWise): (Unnamed Layer* 526) [ElementWise] + (Unnamed Layer* 527) [Activation], Tactic: 1, (Unnamed Layer* 525) [Resize]_output[Float(1,64,160,624)], (Unnamed Layer* 524) [Scale]_output[Float(1,64,160,624)] → (Unnamed Layer* 527) [Activation]_output[Float(1,64,160,624)]
Layer(CaskConvolution): (Unnamed Layer* 528) [Convolution], Tactic: 2775507031594384867, (Unnamed Layer* 527) [Activation]_output[Float(1,64,160,624)] → (Unnamed Layer* 528) [Convolution]_output[Float(1,32,160,624)]
Layer(CaskConvolution): (Unnamed Layer* 340) [Convolution], Tactic: 2775507031594384867, (Unnamed Layer* 339) [Activation]_output[Float(1,64,160,624)] → (Unnamed Layer* 340) [Convolution]_output[Float(1,32,160,624)]
Layer(Reduce): (Unnamed Layer* 342) [Reduce], Tactic: 1, (Unnamed Layer* 341) [Shuffle]_output[Float(1,32,99840)] → (Unnamed Layer* 342) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 343) [ElementWise], Tactic: 1, (Unnamed Layer* 341) [Shuffle]_output[Float(1,32,99840)], (Unnamed Layer* 342) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)]
Layer(Constant): (Unnamed Layer* 348) [Constant], Tactic: 0, → (Unnamed Layer* 348) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 346) [Constant], Tactic: 0, → (Unnamed Layer* 346) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 344) [ElementWise] + (Unnamed Layer* 345) [Reduce], Tactic: 2, (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 345) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 347) [Constant], Tactic: 0, → (Unnamed Layer* 347) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 349) [ElementWise], (Unnamed Layer* 350) [ElementWise]), (Unnamed Layer* 351) [ElementWise]), (Unnamed Layer* 352) [ElementWise]), Tactic: 5, (Unnamed Layer* 345) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 347) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 346) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 348) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 352) [ElementWise]_output[Float(1,32,99840)]
Layer(Scale): (Unnamed Layer* 354) [Scale], Tactic: 0, (Unnamed Layer* 353) [Shuffle]_output[Float(1,32,160,624)] → (Unnamed Layer* 354) [Scale]_output[Float(1,32,160,624)]
Layer(Reduce): (Unnamed Layer* 530) [Reduce], Tactic: 1, (Unnamed Layer* 529) [Shuffle]_output[Float(1,32,99840)] → (Unnamed Layer* 530) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 531) [ElementWise], Tactic: 1, (Unnamed Layer* 529) [Shuffle]_output[Float(1,32,99840)], (Unnamed Layer* 530) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)]
Layer(Constant): (Unnamed Layer* 536) [Constant], Tactic: 0, → (Unnamed Layer* 536) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 534) [Constant], Tactic: 0, → (Unnamed Layer* 534) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 532) [ElementWise] + (Unnamed Layer* 533) [Reduce], Tactic: 2, (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 533) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 535) [Constant], Tactic: 0, → (Unnamed Layer* 535) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 537) [ElementWise], (Unnamed Layer* 538) [ElementWise]), (Unnamed Layer* 539) [ElementWise]), (Unnamed Layer* 540) [ElementWise]), Tactic: 5, (Unnamed Layer* 533) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 535) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 534) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 536) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 540) [ElementWise]_output[Float(1,32,99840)]
Layer(Scale): (Unnamed Layer* 542) [Scale], Tactic: 0, (Unnamed Layer* 541) [Shuffle]_output[Float(1,32,160,624)] → (Unnamed Layer* 542) [Scale]_output[Float(1,32,160,624)]
Layer(Resize): (Unnamed Layer* 558) [Resize], Tactic: 1, (Unnamed Layer* 542) [Scale]_output[Float(1,32,160,624)] → (Unnamed Layer* 558) [Resize]_output[Float(1,32,320,1248)]
Layer(Resize): (Unnamed Layer* 370) [Resize], Tactic: 1, (Unnamed Layer* 354) [Scale]_output[Float(1,32,160,624)] → (Unnamed Layer* 370) [Resize]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 357) [Reduce], Tactic: 1, (Unnamed Layer* 356) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 357) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 358) [ElementWise], Tactic: 1, (Unnamed Layer* 356) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 357) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 363) [Constant], Tactic: 0, → (Unnamed Layer* 363) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 361) [Constant], Tactic: 0, → (Unnamed Layer* 361) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 359) [ElementWise] + (Unnamed Layer* 360) [Reduce], Tactic: 2, (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 360) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 362) [Constant], Tactic: 0, → (Unnamed Layer* 362) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 364) [ElementWise], (Unnamed Layer* 365) [ElementWise]), (Unnamed Layer* 366) [ElementWise]), (Unnamed Layer* 367) [ElementWise]), Tactic: 6, (Unnamed Layer* 360) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 362) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 361) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 363) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 367) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 369) [Scale], Tactic: 0, (Unnamed Layer* 368) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 369) [Scale]_output[Float(1,32,320,1248)]
Layer(ElementWise): (Unnamed Layer* 371) [ElementWise] + (Unnamed Layer* 372) [Activation], Tactic: 1, (Unnamed Layer* 370) [Resize]_output[Float(1,32,320,1248)], (Unnamed Layer* 369) [Scale]_output[Float(1,32,320,1248)] → (Unnamed Layer* 372) [Activation]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 545) [Reduce], Tactic: 1, (Unnamed Layer* 544) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 545) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 546) [ElementWise], Tactic: 1, (Unnamed Layer* 544) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 545) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 551) [Constant], Tactic: 0, → (Unnamed Layer* 551) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 549) [Constant], Tactic: 0, → (Unnamed Layer* 549) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 547) [ElementWise] + (Unnamed Layer* 548) [Reduce], Tactic: 1, (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 548) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 550) [Constant], Tactic: 0, → (Unnamed Layer* 550) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 552) [ElementWise], (Unnamed Layer* 553) [ElementWise]), (Unnamed Layer* 554) [ElementWise]), (Unnamed Layer* 555) [ElementWise]), Tactic: 6, (Unnamed Layer* 548) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 550) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 549) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 551) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 555) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 557) [Scale], Tactic: 0, (Unnamed Layer* 556) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 557) [Scale]_output[Float(1,32,320,1248)]
Layer(ElementWise): (Unnamed Layer* 559) [ElementWise] + (Unnamed Layer* 560) [Activation], Tactic: 1, (Unnamed Layer* 558) [Resize]_output[Float(1,32,320,1248)], (Unnamed Layer* 557) [Scale]_output[Float(1,32,320,1248)] → (Unnamed Layer* 560) [Activation]_output[Float(1,32,320,1248)]
Layer(CudnnConvolution): (Unnamed Layer* 561) [Convolution], Tactic: 6, (Unnamed Layer* 560) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 561) [Convolution]_output[Float(1,32,320,1248)]
Layer(CudnnConvolution): (Unnamed Layer* 373) [Convolution], Tactic: 6, (Unnamed Layer* 372) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 373) [Convolution]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 375) [Reduce], Tactic: 1, (Unnamed Layer* 374) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 375) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 376) [ElementWise], Tactic: 1, (Unnamed Layer* 374) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 375) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 381) [Constant], Tactic: 0, → (Unnamed Layer* 381) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 379) [Constant], Tactic: 0, → (Unnamed Layer* 379) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 377) [ElementWise] + (Unnamed Layer* 378) [Reduce], Tactic: 1, (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 378) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 380) [Constant], Tactic: 0, → (Unnamed Layer* 380) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 382) [ElementWise], (Unnamed Layer* 383) [ElementWise]), (Unnamed Layer* 384) [ElementWise]), (Unnamed Layer* 385) [ElementWise]), Tactic: 6, (Unnamed Layer* 378) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 380) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 379) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 381) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 385) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 387) [Scale] + (Unnamed Layer* 388) [Activation], Tactic: 0, (Unnamed Layer* 386) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 388) [Activation]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 563) [Reduce], Tactic: 1, (Unnamed Layer* 562) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 563) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 564) [ElementWise], Tactic: 1, (Unnamed Layer* 562) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 563) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 569) [Constant], Tactic: 0, → (Unnamed Layer* 569) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 567) [Constant], Tactic: 0, → (Unnamed Layer* 567) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 565) [ElementWise] + (Unnamed Layer* 566) [Reduce], Tactic: 2, (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 566) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 568) [Constant], Tactic: 0, → (Unnamed Layer* 568) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 570) [ElementWise], (Unnamed Layer* 571) [ElementWise]), (Unnamed Layer* 572) [ElementWise]), (Unnamed Layer* 573) [ElementWise]), Tactic: 6, (Unnamed Layer* 566) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 568) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 567) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 569) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 573) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 575) [Scale] + (Unnamed Layer* 576) [Activation], Tactic: 0, (Unnamed Layer* 574) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 576) [Activation]_output[Float(1,32,320,1248)]
Layer(CaskConvolution): (Unnamed Layer* 577) [Convolution], Tactic: -3946921629105938337, (Unnamed Layer* 576) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 577) [Convolution]_output[Float(1,32,320,1248)]
Layer(CaskConvolution): (Unnamed Layer* 389) [Convolution], Tactic: -3946921629105938337, (Unnamed Layer* 388) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 389) [Convolution]_output[Float(1,32,320,1248)]
Layer(Constant): (Unnamed Layer* 616) [Constant], Tactic: 0, → (Unnamed Layer* 616) [Constant]_output[Float(72,1)]
Layer(Constant): (Unnamed Layer* 617) [Constant], Tactic: 0, → (Unnamed Layer* 617) [Constant]_output[Float(72,1)]
Layer(ElementWise): (Unnamed Layer* 618) [ElementWise], Tactic: 1, (Unnamed Layer* 617) [Constant]_output[Float(72,1)], (Unnamed Layer* 616) [Constant]_output[Float(72,1)] → (Unnamed Layer* 618) [ElementWise]_output[Float(72,1)]
Layer(PluginV2): (Unnamed Layer* 620) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 389) [Convolution]_output[Float(1,32,320,1248)], (Unnamed Layer* 577) [Convolution]_output[Float(1,32,320,1248)], (Unnamed Layer* 619) [Shuffle]_output[Float(1,72,1,1)] → (Unnamed Layer* 620) [PluginV2DynamicExt]_output_0[Float(64,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 622) [Convolution], Tactic: 57, (Unnamed Layer* 621) [Shuffle]_output[Float(1,64,72,80,312)] → (Unnamed Layer* 622) [Convolution]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 623) [Constant], Tactic: 0, → (Unnamed Layer* 623) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 624) [Constant], Tactic: 0, → (Unnamed Layer* 624) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 625) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 622) [Convolution]_output[Float(1,32,72,80,312)], (Unnamed Layer* 623) [Constant]_output[Float(32,1)], (Unnamed Layer* 624) [Constant]_output[Float(32,1)] → (Unnamed Layer* 625) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(PointWiseV2): PWN((Unnamed Layer* 626) [Activation]), Tactic: 8, (Unnamed Layer* 625) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)] → (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 627) [Convolution], Tactic: 57, (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 627) [Convolution]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 628) [Constant], Tactic: 0, → (Unnamed Layer* 628) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 629) [Constant], Tactic: 0, → (Unnamed Layer* 629) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 630) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 627) [Convolution]_output[Float(1,32,72,80,312)], (Unnamed Layer* 628) [Constant]_output[Float(32,1)], (Unnamed Layer* 629) [Constant]_output[Float(32,1)] → (Unnamed Layer* 630) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(PointWiseV2): PWN(PWN((Unnamed Layer* 631) [Activation]), (Unnamed Layer* 632) [ElementWise]), Tactic: 8, (Unnamed Layer* 630) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)], (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 633) [Convolution], Tactic: 1, (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 633) [Convolution]_output[Float(1,64,36,40,156)]
Layer(Constant): (Unnamed Layer* 634) [Constant], Tactic: 0, → (Unnamed Layer* 634) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 635) [Constant], Tactic: 0, → (Unnamed Layer* 635) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 636) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 633) [Convolution]_output[Float(1,64,36,40,156)], (Unnamed Layer* 634) [Constant]_output[Float(64,1)], (Unnamed Layer* 635) [Constant]_output[Float(64,1)] → (Unnamed Layer* 636) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(PointWiseV2): PWN((Unnamed Layer* 637) [Activation]), Tactic: 8, (Unnamed Layer* 636) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)] → (Unnamed Layer* 637) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnConvolution): (Unnamed Layer* 638) [Convolution], Tactic: 57, (Unnamed Layer* 637) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 638) [Convolution]_output[Float(1,64,36,40,156)]
Layer(Constant): (Unnamed Layer* 639) [Constant], Tactic: 0, → (Unnamed Layer* 639) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 640) [Constant], Tactic: 0, → (Unnamed Layer* 640) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 641) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 638) [Convolution]_output[Float(1,64,36,40,156)], (Unnamed Layer* 639) [Constant]_output[Float(64,1)], (Unnamed Layer* 640) [Constant]_output[Float(64,1)] → (Unnamed Layer* 641) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(PointWiseV2): PWN((Unnamed Layer* 642) [Activation]), Tactic: 8, (Unnamed Layer* 641) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)] → (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnConvolution): (Unnamed Layer* 643) [Convolution], Tactic: 1, (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 643) [Convolution]_output[Float(1,64,18,20,78)]
Layer(Constant): (Unnamed Layer* 644) [Constant], Tactic: 0, → (Unnamed Layer* 644) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 645) [Constant], Tactic: 0, → (Unnamed Layer* 645) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 646) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 643) [Convolution]_output[Float(1,64,18,20,78)], (Unnamed Layer* 644) [Constant]_output[Float(64,1)], (Unnamed Layer* 645) [Constant]_output[Float(64,1)] → (Unnamed Layer* 646) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)]
Layer(PointWiseV2): PWN((Unnamed Layer* 647) [Activation]), Tactic: 8, (Unnamed Layer* 646) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)] → (Unnamed Layer* 647) [Activation]_output[Float(1,64,18,20,78)]
Layer(CudnnConvolution): (Unnamed Layer* 648) [Convolution], Tactic: 1, (Unnamed Layer* 647) [Activation]_output[Float(1,64,18,20,78)] → (Unnamed Layer* 648) [Convolution]_output[Float(1,64,18,20,78)]
Layer(Constant): (Unnamed Layer* 649) [Constant], Tactic: 0, → (Unnamed Layer* 649) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 650) [Constant], Tactic: 0, → (Unnamed Layer* 650) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 651) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 648) [Convolution]_output[Float(1,64,18,20,78)], (Unnamed Layer* 649) [Constant]_output[Float(64,1)], (Unnamed Layer* 650) [Constant]_output[Float(64,1)] → (Unnamed Layer* 651) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)]
Layer(PointWiseV2): PWN((Unnamed Layer* 652) [Activation]), Tactic: 8, (Unnamed Layer* 651) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)] → (Unnamed Layer* 652) [Activation]_output[Float(1,64,18,20,78)]
Layer(CudnnDeconvolution): (Unnamed Layer* 653) [Deconvolution], Tactic: 0, (Unnamed Layer* 652) [Activation]_output[Float(1,64,18,20,78)] → (Unnamed Layer* 653) [Deconvolution]_output[Float(1,64,35,39,155)]
Layer(Constant): (Unnamed Layer* 654) [Constant], Tactic: 0, → (Unnamed Layer* 654) [Constant]_output[Float(1,836160)]
Layer(Reformat): (Unnamed Layer* 655) [Shuffle]_output copy, Tactic: 0, (Unnamed Layer* 655) [Shuffle]_output[Float(1,13540800)] → (Unnamed Layer* 656) [Concatenation]_output[Float(1,13540800)]
Layer(Reformat): (Unnamed Layer* 654) [Constant]_output copy, Tactic: 0, (Unnamed Layer* 654) [Constant]_output[Float(1,836160)] → (Unnamed Layer* 656) [Concatenation]_output[Float(1,836160)]
Layer(Constant): (Unnamed Layer* 658) [Constant], Tactic: 0, → (Unnamed Layer* 658) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 659) [Constant], Tactic: 0, → (Unnamed Layer* 659) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 660) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 657) [Shuffle]_output[Float(1,64,36,40,156)], (Unnamed Layer* 658) [Constant]_output[Float(64,1)], (Unnamed Layer* 659) [Constant]_output[Float(64,1)] → (Unnamed Layer* 660) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(ElementWise): (Unnamed Layer* 661) [ElementWise] + (Unnamed Layer* 662) [Activation], Tactic: 1, (Unnamed Layer* 660) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)], (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 662) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnDeconvolution): (Unnamed Layer* 663) [Deconvolution], Tactic: 0, (Unnamed Layer* 662) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 663) [Deconvolution]_output[Float(1,32,71,79,311)]
Layer(Constant): (Unnamed Layer* 664) [Constant], Tactic: 0, → (Unnamed Layer* 664) [Constant]_output[Float(1,1687072)]
Layer(Reformat): (Unnamed Layer* 665) [Shuffle]_output copy, Tactic: 0, (Unnamed Layer* 665) [Shuffle]_output[Float(1,55820768)] → (Unnamed Layer* 666) [Concatenation]_output[Float(1,55820768)]
Layer(Reformat): (Unnamed Layer* 664) [Constant]_output copy, Tactic: 0, (Unnamed Layer* 664) [Constant]_output[Float(1,1687072)] → (Unnamed Layer* 666) [Concatenation]_output[Float(1,1687072)]
Layer(Constant): (Unnamed Layer* 668) [Constant], Tactic: 0, → (Unnamed Layer* 668) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 669) [Constant], Tactic: 0, → (Unnamed Layer* 669) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 670) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 667) [Shuffle]_output[Float(1,32,72,80,312)], (Unnamed Layer* 668) [Constant]_output[Float(32,1)], (Unnamed Layer* 669) [Constant]_output[Float(32,1)] → (Unnamed Layer* 670) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(ElementWise): (Unnamed Layer* 671) [ElementWise], Tactic: 1, (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)], (Unnamed Layer* 670) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)] → (Unnamed Layer* 671) [ElementWise]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 726) [Constant], Tactic: 0, → (Unnamed Layer* 726) [Constant]_output[Float(1,20,304,288,3)]
Layer(PluginV2): (Unnamed Layer* 727) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 671) [ElementWise]_output[Float(1,32,72,80,312)], (Unnamed Layer* 726) [Constant]_output[Float(1,20,304,288,3)] → (Unnamed Layer* 727) [PluginV2DynamicExt]_output_0[Float(1,32,20,304,288)]
Layer(Myelin): {ForeignNode[(Unnamed Layer* 730) [ElementWise]]}, Tactic: 0, (Unnamed Layer* 727) [PluginV2DynamicExt]_output_0[Float(1,32,20,304,288)], (Unnamed Layer* 729) [Identity]_output[Float(1,1,20,304,288)] → Box[Float(1,32,20,304,288)]
[TensorRT] INFO: [MemUsageSnapshot] Builder end: CPU 1563 MiB, GPU 9954 MiB
[TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1563, GPU 9955 (MiB)
[TensorRT] INFO: Loaded engine size: 157 MB
[TensorRT] INFO: [MemUsageSnapshot] deserializeCudaEngine begin: CPU 1720 MiB, GPU 10113 MiB
[TensorRT] VERBOSE: Using cublas a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1726, GPU 10256 (MiB)
[TensorRT] VERBOSE: Using cuDNN as a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 1726, GPU 10266 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1726, GPU 10261 (MiB)
[TensorRT] VERBOSE: Deserialization required 512539 microseconds.
[TensorRT] INFO: [MemUsageSnapshot] deserializeCudaEngine end: CPU 1726 MiB, GPU 10261 MiB
[TensorRT] INFO: [MemUsageSnapshot] ExecutionContext creation begin: CPU 1532 MiB, GPU 9842 MiB
[TensorRT] VERBOSE: Using cublas a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1532, GPU 9842 (MiB)
[TensorRT] VERBOSE: Using cuDNN as a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +0, GPU +7, now: CPU 1532, GPU 9849 (MiB)
[TensorRT] VERBOSE: Total per-runner device memory is 90608640
[TensorRT] VERBOSE: Total per-runner host memory is 109936
[TensorRT] VERBOSE: Allocated activation device memory of size 1228143616
[TensorRT] VERBOSE: myelinAllocCb allocated GPU (data-constants) 8 bytes at 0x226cc7500.
[TensorRT] INFO: [MemUsageSnapshot] ExecutionContext creation end: CPU 1537 MiB, GPU 11109 MiB
[TensorRT] VERBOSE: myelinAllocCb allocated GPU 63037440 bytes at 0x22ddea000.
[TensorRT] ERROR: 2: [pluginV2DynamicExtRunner.cpp::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed.)

Inference code snippet

def do_inference(context, host_in, host_out):
engine = context.engine
#assert engine.num_bindings == 6
bindings =
bytes = 0

stream = cuda.Stream()
for i in range(len(host_in)):
devide_in = cuda.mem_alloc(host_in[i].nbytes)
cuda.memcpy_htod_async(devide_in, host_in[i], stream)
bindings.append(int(devide_in))
devide_out = cuda.mem_alloc(host_out.nbytes)
bindings.append(int(devide_out))
context.execute_async(bindings=bindings, stream_handle=stream.handle)
cuda.memcpy_dtoh_async(host_out, devide_out, stream)
stream.synchronize()

if name==“main”:
runtime = trt.Runtime(TRT_LOGGER)

assert runtime

with open(ENGINE_PATH, “rb”) as f:
engine = runtime.deserialize_cuda_engine(f.read())
assert engine

context = engine.create_execution_context()
assert context
BATCH_SIZE = 1
image_size = IMAGE_BATCH_SIZE * IMAGE_CHANNELS * IMAGE_HEIGHT * IMAGE_WIDTH

host_in =

left_image = cuda.pagelocked_empty(image_size,
dtype=np.float32)
right_image = cuda.pagelocked_empty(image_size,
dtype=np.float32)
calib = cuda.pagelocked_empty(BATCH_SIZE * 3 * 4, dtype=np.float32)

data_left = cv2.imread("./liga/test/left/000000.png")
data_right = cv2.imread("./liga/test/right/000000.png")
calib_project = np.random.rand(12)
data_left = cv2.resize(data_left, (IMAGE_HEIGHT, IMAGE_WIDTH)).flatten()
data_right = cv2.resize(data_right, (IMAGE_HEIGHT, IMAGE_WIDTH)).flatten()

np.copyto(left_image, data_left.ravel())
np.copyto(right_image, data_right.ravel())
np.copyto(calib, calib_project.ravel())
host_in.append(left_image)
host_in.append(right_image)
host_in.append(calib)

host_out = cuda.pagelocked_empty(BATCH_SIZE * 32 * 20 * 304*288, dtype=np.float32)

do_inference(context, host_in, host_out)

print(f’Output: \n{host_out[:5000]}\n{host_out[-10:]}’)

Environment

TensorRT Version: 8.0.1
GPU Type: AGX Xavier
Nvidia Driver Version:
CUDA Version: 10.2
CUDNN Version: 8.2
Operating System + Version: JetPack4.6
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Hi,
Please refer to below links related custom plugin implementation and sample:

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

Hi,
Thank you for your reply. The 3d grid sample plugin is developed using IPluginV2DynamicExt interface. And when I call the plugin independently, it can be implemented successfully. The error occur when I using the grid sample result to do elementwise operation. So I wonder is there any limitation of [N,C,D,H,W] tensor operation?

Thank you!

Could you please share with us the issue repro onnx model/scripts to try from our end for better debugging.

Thank you.