Description
Hi, I have build a model that has three inputs with tensorrt python api layer by layer, and I develop a 3d grid sample plugin to build voxel. Everything runs well when building an engine, but when the engine is used fo inference there’s an error thrown. But when the plugin can be successfully implemented independently. Is anyone knows what’s the problem.
Error log
Layer(Constant): (Unnamed Layer* 329) [Constant], Tactic: 0, → (Unnamed Layer* 329) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 331) [ElementWise], (Unnamed Layer* 332) [ElementWise]), (Unnamed Layer* 333) [ElementWise]), (Unnamed Layer* 334) [ElementWise]), Tactic: 6, (Unnamed Layer* 327) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 329) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 328) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 330) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 325) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 334) [ElementWise]_output[Float(1,32,199680)]
Layer(Scale): (Unnamed Layer* 336) [Scale], Tactic: 0, (Unnamed Layer* 335) [Shuffle]_output[Float(1,64,160,624)] → (Unnamed Layer* 336) [Scale]_output[Float(1,64,160,624)]
Layer(ElementWise): (Unnamed Layer* 338) [ElementWise] + (Unnamed Layer* 339) [Activation], Tactic: 1, (Unnamed Layer* 337) [Resize]_output[Float(1,64,160,624)], (Unnamed Layer* 336) [Scale]_output[Float(1,64,160,624)] → (Unnamed Layer* 339) [Activation]_output[Float(1,64,160,624)]
Layer(Reduce): (Unnamed Layer* 512) [Reduce], Tactic: 1, (Unnamed Layer* 511) [Shuffle]_output[Float(1,32,199680)] → (Unnamed Layer* 512) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 513) [ElementWise], Tactic: 1, (Unnamed Layer* 511) [Shuffle]_output[Float(1,32,199680)], (Unnamed Layer* 512) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)]
Layer(Constant): (Unnamed Layer* 518) [Constant], Tactic: 0, → (Unnamed Layer* 518) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 516) [Constant], Tactic: 0, → (Unnamed Layer* 516) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 514) [ElementWise] + (Unnamed Layer* 515) [Reduce], Tactic: 2, (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 515) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 517) [Constant], Tactic: 0, → (Unnamed Layer* 517) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 519) [ElementWise], (Unnamed Layer* 520) [ElementWise]), (Unnamed Layer* 521) [ElementWise]), (Unnamed Layer* 522) [ElementWise]), Tactic: 6, (Unnamed Layer* 515) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 517) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 516) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 518) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 513) [ElementWise]_output[Float(1,32,199680)] → (Unnamed Layer* 522) [ElementWise]_output[Float(1,32,199680)]
Layer(Scale): (Unnamed Layer* 524) [Scale], Tactic: 0, (Unnamed Layer* 523) [Shuffle]_output[Float(1,64,160,624)] → (Unnamed Layer* 524) [Scale]_output[Float(1,64,160,624)]
Layer(ElementWise): (Unnamed Layer* 526) [ElementWise] + (Unnamed Layer* 527) [Activation], Tactic: 1, (Unnamed Layer* 525) [Resize]_output[Float(1,64,160,624)], (Unnamed Layer* 524) [Scale]_output[Float(1,64,160,624)] → (Unnamed Layer* 527) [Activation]_output[Float(1,64,160,624)]
Layer(CaskConvolution): (Unnamed Layer* 528) [Convolution], Tactic: 2775507031594384867, (Unnamed Layer* 527) [Activation]_output[Float(1,64,160,624)] → (Unnamed Layer* 528) [Convolution]_output[Float(1,32,160,624)]
Layer(CaskConvolution): (Unnamed Layer* 340) [Convolution], Tactic: 2775507031594384867, (Unnamed Layer* 339) [Activation]_output[Float(1,64,160,624)] → (Unnamed Layer* 340) [Convolution]_output[Float(1,32,160,624)]
Layer(Reduce): (Unnamed Layer* 342) [Reduce], Tactic: 1, (Unnamed Layer* 341) [Shuffle]_output[Float(1,32,99840)] → (Unnamed Layer* 342) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 343) [ElementWise], Tactic: 1, (Unnamed Layer* 341) [Shuffle]_output[Float(1,32,99840)], (Unnamed Layer* 342) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)]
Layer(Constant): (Unnamed Layer* 348) [Constant], Tactic: 0, → (Unnamed Layer* 348) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 346) [Constant], Tactic: 0, → (Unnamed Layer* 346) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 344) [ElementWise] + (Unnamed Layer* 345) [Reduce], Tactic: 2, (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 345) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 347) [Constant], Tactic: 0, → (Unnamed Layer* 347) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 349) [ElementWise], (Unnamed Layer* 350) [ElementWise]), (Unnamed Layer* 351) [ElementWise]), (Unnamed Layer* 352) [ElementWise]), Tactic: 5, (Unnamed Layer* 345) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 347) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 346) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 348) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 343) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 352) [ElementWise]_output[Float(1,32,99840)]
Layer(Scale): (Unnamed Layer* 354) [Scale], Tactic: 0, (Unnamed Layer* 353) [Shuffle]_output[Float(1,32,160,624)] → (Unnamed Layer* 354) [Scale]_output[Float(1,32,160,624)]
Layer(Reduce): (Unnamed Layer* 530) [Reduce], Tactic: 1, (Unnamed Layer* 529) [Shuffle]_output[Float(1,32,99840)] → (Unnamed Layer* 530) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 531) [ElementWise], Tactic: 1, (Unnamed Layer* 529) [Shuffle]_output[Float(1,32,99840)], (Unnamed Layer* 530) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)]
Layer(Constant): (Unnamed Layer* 536) [Constant], Tactic: 0, → (Unnamed Layer* 536) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 534) [Constant], Tactic: 0, → (Unnamed Layer* 534) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 532) [ElementWise] + (Unnamed Layer* 533) [Reduce], Tactic: 2, (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 533) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 535) [Constant], Tactic: 0, → (Unnamed Layer* 535) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 537) [ElementWise], (Unnamed Layer* 538) [ElementWise]), (Unnamed Layer* 539) [ElementWise]), (Unnamed Layer* 540) [ElementWise]), Tactic: 5, (Unnamed Layer* 533) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 535) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 534) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 536) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 531) [ElementWise]_output[Float(1,32,99840)] → (Unnamed Layer* 540) [ElementWise]_output[Float(1,32,99840)]
Layer(Scale): (Unnamed Layer* 542) [Scale], Tactic: 0, (Unnamed Layer* 541) [Shuffle]_output[Float(1,32,160,624)] → (Unnamed Layer* 542) [Scale]_output[Float(1,32,160,624)]
Layer(Resize): (Unnamed Layer* 558) [Resize], Tactic: 1, (Unnamed Layer* 542) [Scale]_output[Float(1,32,160,624)] → (Unnamed Layer* 558) [Resize]_output[Float(1,32,320,1248)]
Layer(Resize): (Unnamed Layer* 370) [Resize], Tactic: 1, (Unnamed Layer* 354) [Scale]_output[Float(1,32,160,624)] → (Unnamed Layer* 370) [Resize]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 357) [Reduce], Tactic: 1, (Unnamed Layer* 356) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 357) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 358) [ElementWise], Tactic: 1, (Unnamed Layer* 356) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 357) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 363) [Constant], Tactic: 0, → (Unnamed Layer* 363) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 361) [Constant], Tactic: 0, → (Unnamed Layer* 361) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 359) [ElementWise] + (Unnamed Layer* 360) [Reduce], Tactic: 2, (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 360) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 362) [Constant], Tactic: 0, → (Unnamed Layer* 362) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 364) [ElementWise], (Unnamed Layer* 365) [ElementWise]), (Unnamed Layer* 366) [ElementWise]), (Unnamed Layer* 367) [ElementWise]), Tactic: 6, (Unnamed Layer* 360) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 362) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 361) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 363) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 358) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 367) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 369) [Scale], Tactic: 0, (Unnamed Layer* 368) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 369) [Scale]_output[Float(1,32,320,1248)]
Layer(ElementWise): (Unnamed Layer* 371) [ElementWise] + (Unnamed Layer* 372) [Activation], Tactic: 1, (Unnamed Layer* 370) [Resize]_output[Float(1,32,320,1248)], (Unnamed Layer* 369) [Scale]_output[Float(1,32,320,1248)] → (Unnamed Layer* 372) [Activation]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 545) [Reduce], Tactic: 1, (Unnamed Layer* 544) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 545) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 546) [ElementWise], Tactic: 1, (Unnamed Layer* 544) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 545) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 551) [Constant], Tactic: 0, → (Unnamed Layer* 551) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 549) [Constant], Tactic: 0, → (Unnamed Layer* 549) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 547) [ElementWise] + (Unnamed Layer* 548) [Reduce], Tactic: 1, (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 548) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 550) [Constant], Tactic: 0, → (Unnamed Layer* 550) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 552) [ElementWise], (Unnamed Layer* 553) [ElementWise]), (Unnamed Layer* 554) [ElementWise]), (Unnamed Layer* 555) [ElementWise]), Tactic: 6, (Unnamed Layer* 548) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 550) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 549) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 551) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 546) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 555) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 557) [Scale], Tactic: 0, (Unnamed Layer* 556) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 557) [Scale]_output[Float(1,32,320,1248)]
Layer(ElementWise): (Unnamed Layer* 559) [ElementWise] + (Unnamed Layer* 560) [Activation], Tactic: 1, (Unnamed Layer* 558) [Resize]_output[Float(1,32,320,1248)], (Unnamed Layer* 557) [Scale]_output[Float(1,32,320,1248)] → (Unnamed Layer* 560) [Activation]_output[Float(1,32,320,1248)]
Layer(CudnnConvolution): (Unnamed Layer* 561) [Convolution], Tactic: 6, (Unnamed Layer* 560) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 561) [Convolution]_output[Float(1,32,320,1248)]
Layer(CudnnConvolution): (Unnamed Layer* 373) [Convolution], Tactic: 6, (Unnamed Layer* 372) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 373) [Convolution]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 375) [Reduce], Tactic: 1, (Unnamed Layer* 374) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 375) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 376) [ElementWise], Tactic: 1, (Unnamed Layer* 374) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 375) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 381) [Constant], Tactic: 0, → (Unnamed Layer* 381) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 379) [Constant], Tactic: 0, → (Unnamed Layer* 379) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 377) [ElementWise] + (Unnamed Layer* 378) [Reduce], Tactic: 1, (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 378) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 380) [Constant], Tactic: 0, → (Unnamed Layer* 380) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 382) [ElementWise], (Unnamed Layer* 383) [ElementWise]), (Unnamed Layer* 384) [ElementWise]), (Unnamed Layer* 385) [ElementWise]), Tactic: 6, (Unnamed Layer* 378) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 380) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 379) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 381) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 376) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 385) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 387) [Scale] + (Unnamed Layer* 388) [Activation], Tactic: 0, (Unnamed Layer* 386) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 388) [Activation]_output[Float(1,32,320,1248)]
Layer(Reduce): (Unnamed Layer* 563) [Reduce], Tactic: 1, (Unnamed Layer* 562) [Shuffle]_output[Float(1,32,399360)] → (Unnamed Layer* 563) [Reduce]_output[Float(1,32,1)]
Layer(ElementWise): (Unnamed Layer* 564) [ElementWise], Tactic: 1, (Unnamed Layer* 562) [Shuffle]_output[Float(1,32,399360)], (Unnamed Layer* 563) [Reduce]_output[Float(1,32,1)] → (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)]
Layer(Constant): (Unnamed Layer* 569) [Constant], Tactic: 0, → (Unnamed Layer* 569) [Constant]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 567) [Constant], Tactic: 0, → (Unnamed Layer* 567) [Constant]_output[Float(1,32,1)]
Layer(Reduce): (Unnamed Layer* 565) [ElementWise] + (Unnamed Layer* 566) [Reduce], Tactic: 2, (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 566) [Reduce]_output[Float(1,32,1)]
Layer(Constant): (Unnamed Layer* 568) [Constant], Tactic: 0, → (Unnamed Layer* 568) [Constant]_output[Float(1,32,1)]
Layer(PointWiseV2): PWN(PWN(PWN((Unnamed Layer* 570) [ElementWise], (Unnamed Layer* 571) [ElementWise]), (Unnamed Layer* 572) [ElementWise]), (Unnamed Layer* 573) [ElementWise]), Tactic: 6, (Unnamed Layer* 566) [Reduce]_output[Float(1,32,1)], (Unnamed Layer* 568) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 567) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 569) [Constant]_output[Float(1,32,1)], (Unnamed Layer* 564) [ElementWise]_output[Float(1,32,399360)] → (Unnamed Layer* 573) [ElementWise]_output[Float(1,32,399360)]
Layer(Scale): (Unnamed Layer* 575) [Scale] + (Unnamed Layer* 576) [Activation], Tactic: 0, (Unnamed Layer* 574) [Shuffle]_output[Float(1,32,320,1248)] → (Unnamed Layer* 576) [Activation]_output[Float(1,32,320,1248)]
Layer(CaskConvolution): (Unnamed Layer* 577) [Convolution], Tactic: -3946921629105938337, (Unnamed Layer* 576) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 577) [Convolution]_output[Float(1,32,320,1248)]
Layer(CaskConvolution): (Unnamed Layer* 389) [Convolution], Tactic: -3946921629105938337, (Unnamed Layer* 388) [Activation]_output[Float(1,32,320,1248)] → (Unnamed Layer* 389) [Convolution]_output[Float(1,32,320,1248)]
Layer(Constant): (Unnamed Layer* 616) [Constant], Tactic: 0, → (Unnamed Layer* 616) [Constant]_output[Float(72,1)]
Layer(Constant): (Unnamed Layer* 617) [Constant], Tactic: 0, → (Unnamed Layer* 617) [Constant]_output[Float(72,1)]
Layer(ElementWise): (Unnamed Layer* 618) [ElementWise], Tactic: 1, (Unnamed Layer* 617) [Constant]_output[Float(72,1)], (Unnamed Layer* 616) [Constant]_output[Float(72,1)] → (Unnamed Layer* 618) [ElementWise]_output[Float(72,1)]
Layer(PluginV2): (Unnamed Layer* 620) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 389) [Convolution]_output[Float(1,32,320,1248)], (Unnamed Layer* 577) [Convolution]_output[Float(1,32,320,1248)], (Unnamed Layer* 619) [Shuffle]_output[Float(1,72,1,1)] → (Unnamed Layer* 620) [PluginV2DynamicExt]_output_0[Float(64,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 622) [Convolution], Tactic: 57, (Unnamed Layer* 621) [Shuffle]_output[Float(1,64,72,80,312)] → (Unnamed Layer* 622) [Convolution]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 623) [Constant], Tactic: 0, → (Unnamed Layer* 623) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 624) [Constant], Tactic: 0, → (Unnamed Layer* 624) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 625) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 622) [Convolution]_output[Float(1,32,72,80,312)], (Unnamed Layer* 623) [Constant]_output[Float(32,1)], (Unnamed Layer* 624) [Constant]_output[Float(32,1)] → (Unnamed Layer* 625) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(PointWiseV2): PWN((Unnamed Layer* 626) [Activation]), Tactic: 8, (Unnamed Layer* 625) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)] → (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 627) [Convolution], Tactic: 57, (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 627) [Convolution]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 628) [Constant], Tactic: 0, → (Unnamed Layer* 628) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 629) [Constant], Tactic: 0, → (Unnamed Layer* 629) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 630) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 627) [Convolution]_output[Float(1,32,72,80,312)], (Unnamed Layer* 628) [Constant]_output[Float(32,1)], (Unnamed Layer* 629) [Constant]_output[Float(32,1)] → (Unnamed Layer* 630) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(PointWiseV2): PWN(PWN((Unnamed Layer* 631) [Activation]), (Unnamed Layer* 632) [ElementWise]), Tactic: 8, (Unnamed Layer* 630) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)], (Unnamed Layer* 626) [Activation]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)]
Layer(CudnnConvolution): (Unnamed Layer* 633) [Convolution], Tactic: 1, (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)] → (Unnamed Layer* 633) [Convolution]_output[Float(1,64,36,40,156)]
Layer(Constant): (Unnamed Layer* 634) [Constant], Tactic: 0, → (Unnamed Layer* 634) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 635) [Constant], Tactic: 0, → (Unnamed Layer* 635) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 636) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 633) [Convolution]_output[Float(1,64,36,40,156)], (Unnamed Layer* 634) [Constant]_output[Float(64,1)], (Unnamed Layer* 635) [Constant]_output[Float(64,1)] → (Unnamed Layer* 636) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(PointWiseV2): PWN((Unnamed Layer* 637) [Activation]), Tactic: 8, (Unnamed Layer* 636) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)] → (Unnamed Layer* 637) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnConvolution): (Unnamed Layer* 638) [Convolution], Tactic: 57, (Unnamed Layer* 637) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 638) [Convolution]_output[Float(1,64,36,40,156)]
Layer(Constant): (Unnamed Layer* 639) [Constant], Tactic: 0, → (Unnamed Layer* 639) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 640) [Constant], Tactic: 0, → (Unnamed Layer* 640) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 641) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 638) [Convolution]_output[Float(1,64,36,40,156)], (Unnamed Layer* 639) [Constant]_output[Float(64,1)], (Unnamed Layer* 640) [Constant]_output[Float(64,1)] → (Unnamed Layer* 641) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(PointWiseV2): PWN((Unnamed Layer* 642) [Activation]), Tactic: 8, (Unnamed Layer* 641) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)] → (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnConvolution): (Unnamed Layer* 643) [Convolution], Tactic: 1, (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 643) [Convolution]_output[Float(1,64,18,20,78)]
Layer(Constant): (Unnamed Layer* 644) [Constant], Tactic: 0, → (Unnamed Layer* 644) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 645) [Constant], Tactic: 0, → (Unnamed Layer* 645) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 646) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 643) [Convolution]_output[Float(1,64,18,20,78)], (Unnamed Layer* 644) [Constant]_output[Float(64,1)], (Unnamed Layer* 645) [Constant]_output[Float(64,1)] → (Unnamed Layer* 646) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)]
Layer(PointWiseV2): PWN((Unnamed Layer* 647) [Activation]), Tactic: 8, (Unnamed Layer* 646) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)] → (Unnamed Layer* 647) [Activation]_output[Float(1,64,18,20,78)]
Layer(CudnnConvolution): (Unnamed Layer* 648) [Convolution], Tactic: 1, (Unnamed Layer* 647) [Activation]_output[Float(1,64,18,20,78)] → (Unnamed Layer* 648) [Convolution]_output[Float(1,64,18,20,78)]
Layer(Constant): (Unnamed Layer* 649) [Constant], Tactic: 0, → (Unnamed Layer* 649) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 650) [Constant], Tactic: 0, → (Unnamed Layer* 650) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 651) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 648) [Convolution]_output[Float(1,64,18,20,78)], (Unnamed Layer* 649) [Constant]_output[Float(64,1)], (Unnamed Layer* 650) [Constant]_output[Float(64,1)] → (Unnamed Layer* 651) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)]
Layer(PointWiseV2): PWN((Unnamed Layer* 652) [Activation]), Tactic: 8, (Unnamed Layer* 651) [PluginV2DynamicExt]_output_0[Float(1,64,18,20,78)] → (Unnamed Layer* 652) [Activation]_output[Float(1,64,18,20,78)]
Layer(CudnnDeconvolution): (Unnamed Layer* 653) [Deconvolution], Tactic: 0, (Unnamed Layer* 652) [Activation]_output[Float(1,64,18,20,78)] → (Unnamed Layer* 653) [Deconvolution]_output[Float(1,64,35,39,155)]
Layer(Constant): (Unnamed Layer* 654) [Constant], Tactic: 0, → (Unnamed Layer* 654) [Constant]_output[Float(1,836160)]
Layer(Reformat): (Unnamed Layer* 655) [Shuffle]_output copy, Tactic: 0, (Unnamed Layer* 655) [Shuffle]_output[Float(1,13540800)] → (Unnamed Layer* 656) [Concatenation]_output[Float(1,13540800)]
Layer(Reformat): (Unnamed Layer* 654) [Constant]_output copy, Tactic: 0, (Unnamed Layer* 654) [Constant]_output[Float(1,836160)] → (Unnamed Layer* 656) [Concatenation]_output[Float(1,836160)]
Layer(Constant): (Unnamed Layer* 658) [Constant], Tactic: 0, → (Unnamed Layer* 658) [Constant]_output[Float(64,1)]
Layer(Constant): (Unnamed Layer* 659) [Constant], Tactic: 0, → (Unnamed Layer* 659) [Constant]_output[Float(64,1)]
Layer(PluginV2): (Unnamed Layer* 660) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 657) [Shuffle]_output[Float(1,64,36,40,156)], (Unnamed Layer* 658) [Constant]_output[Float(64,1)], (Unnamed Layer* 659) [Constant]_output[Float(64,1)] → (Unnamed Layer* 660) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)]
Layer(ElementWise): (Unnamed Layer* 661) [ElementWise] + (Unnamed Layer* 662) [Activation], Tactic: 1, (Unnamed Layer* 660) [PluginV2DynamicExt]_output_0[Float(1,64,36,40,156)], (Unnamed Layer* 642) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 662) [Activation]_output[Float(1,64,36,40,156)]
Layer(CudnnDeconvolution): (Unnamed Layer* 663) [Deconvolution], Tactic: 0, (Unnamed Layer* 662) [Activation]_output[Float(1,64,36,40,156)] → (Unnamed Layer* 663) [Deconvolution]_output[Float(1,32,71,79,311)]
Layer(Constant): (Unnamed Layer* 664) [Constant], Tactic: 0, → (Unnamed Layer* 664) [Constant]_output[Float(1,1687072)]
Layer(Reformat): (Unnamed Layer* 665) [Shuffle]_output copy, Tactic: 0, (Unnamed Layer* 665) [Shuffle]_output[Float(1,55820768)] → (Unnamed Layer* 666) [Concatenation]_output[Float(1,55820768)]
Layer(Reformat): (Unnamed Layer* 664) [Constant]_output copy, Tactic: 0, (Unnamed Layer* 664) [Constant]_output[Float(1,1687072)] → (Unnamed Layer* 666) [Concatenation]_output[Float(1,1687072)]
Layer(Constant): (Unnamed Layer* 668) [Constant], Tactic: 0, → (Unnamed Layer* 668) [Constant]_output[Float(32,1)]
Layer(Constant): (Unnamed Layer* 669) [Constant], Tactic: 0, → (Unnamed Layer* 669) [Constant]_output[Float(32,1)]
Layer(PluginV2): (Unnamed Layer* 670) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 667) [Shuffle]_output[Float(1,32,72,80,312)], (Unnamed Layer* 668) [Constant]_output[Float(32,1)], (Unnamed Layer* 669) [Constant]_output[Float(32,1)] → (Unnamed Layer* 670) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)]
Layer(ElementWise): (Unnamed Layer* 671) [ElementWise], Tactic: 1, (Unnamed Layer* 632) [ElementWise]_output[Float(1,32,72,80,312)], (Unnamed Layer* 670) [PluginV2DynamicExt]_output_0[Float(1,32,72,80,312)] → (Unnamed Layer* 671) [ElementWise]_output[Float(1,32,72,80,312)]
Layer(Constant): (Unnamed Layer* 726) [Constant], Tactic: 0, → (Unnamed Layer* 726) [Constant]_output[Float(1,20,304,288,3)]
Layer(PluginV2): (Unnamed Layer* 727) [PluginV2DynamicExt], Tactic: 0, (Unnamed Layer* 671) [ElementWise]_output[Float(1,32,72,80,312)], (Unnamed Layer* 726) [Constant]_output[Float(1,20,304,288,3)] → (Unnamed Layer* 727) [PluginV2DynamicExt]_output_0[Float(1,32,20,304,288)]
Layer(Myelin): {ForeignNode[(Unnamed Layer* 730) [ElementWise]]}, Tactic: 0, (Unnamed Layer* 727) [PluginV2DynamicExt]_output_0[Float(1,32,20,304,288)], (Unnamed Layer* 729) [Identity]_output[Float(1,1,20,304,288)] → Box[Float(1,32,20,304,288)]
[TensorRT] INFO: [MemUsageSnapshot] Builder end: CPU 1563 MiB, GPU 9954 MiB
[TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1563, GPU 9955 (MiB)
[TensorRT] INFO: Loaded engine size: 157 MB
[TensorRT] INFO: [MemUsageSnapshot] deserializeCudaEngine begin: CPU 1720 MiB, GPU 10113 MiB
[TensorRT] VERBOSE: Using cublas a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1726, GPU 10256 (MiB)
[TensorRT] VERBOSE: Using cuDNN as a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 1726, GPU 10266 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1726, GPU 10261 (MiB)
[TensorRT] VERBOSE: Deserialization required 512539 microseconds.
[TensorRT] INFO: [MemUsageSnapshot] deserializeCudaEngine end: CPU 1726 MiB, GPU 10261 MiB
[TensorRT] INFO: [MemUsageSnapshot] ExecutionContext creation begin: CPU 1532 MiB, GPU 9842 MiB
[TensorRT] VERBOSE: Using cublas a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1532, GPU 9842 (MiB)
[TensorRT] VERBOSE: Using cuDNN as a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +0, GPU +7, now: CPU 1532, GPU 9849 (MiB)
[TensorRT] VERBOSE: Total per-runner device memory is 90608640
[TensorRT] VERBOSE: Total per-runner host memory is 109936
[TensorRT] VERBOSE: Allocated activation device memory of size 1228143616
[TensorRT] VERBOSE: myelinAllocCb allocated GPU (data-constants) 8 bytes at 0x226cc7500.
[TensorRT] INFO: [MemUsageSnapshot] ExecutionContext creation end: CPU 1537 MiB, GPU 11109 MiB
[TensorRT] VERBOSE: myelinAllocCb allocated GPU 63037440 bytes at 0x22ddea000.
[TensorRT] ERROR: 2: [pluginV2DynamicExtRunner.cpp::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed.)
Inference code snippet
def do_inference(context, host_in, host_out):
engine = context.engine
#assert engine.num_bindings == 6
bindings =
bytes = 0
stream = cuda.Stream()
for i in range(len(host_in)):
devide_in = cuda.mem_alloc(host_in[i].nbytes)
cuda.memcpy_htod_async(devide_in, host_in[i], stream)
bindings.append(int(devide_in))
devide_out = cuda.mem_alloc(host_out.nbytes)
bindings.append(int(devide_out))
context.execute_async(bindings=bindings, stream_handle=stream.handle)
cuda.memcpy_dtoh_async(host_out, devide_out, stream)
stream.synchronize()
if name==“main”:
runtime = trt.Runtime(TRT_LOGGER)
assert runtime
with open(ENGINE_PATH, “rb”) as f:
engine = runtime.deserialize_cuda_engine(f.read())
assert engine
context = engine.create_execution_context()
assert context
BATCH_SIZE = 1
image_size = IMAGE_BATCH_SIZE * IMAGE_CHANNELS * IMAGE_HEIGHT * IMAGE_WIDTH
host_in =
left_image = cuda.pagelocked_empty(image_size,
dtype=np.float32)
right_image = cuda.pagelocked_empty(image_size,
dtype=np.float32)
calib = cuda.pagelocked_empty(BATCH_SIZE * 3 * 4, dtype=np.float32)
data_left = cv2.imread(“./liga/test/left/000000.png”)
data_right = cv2.imread(“./liga/test/right/000000.png”)
calib_project = np.random.rand(12)
data_left = cv2.resize(data_left, (IMAGE_HEIGHT, IMAGE_WIDTH)).flatten()
data_right = cv2.resize(data_right, (IMAGE_HEIGHT, IMAGE_WIDTH)).flatten()
np.copyto(left_image, data_left.ravel())
np.copyto(right_image, data_right.ravel())
np.copyto(calib, calib_project.ravel())
host_in.append(left_image)
host_in.append(right_image)
host_in.append(calib)
host_out = cuda.pagelocked_empty(BATCH_SIZE * 32 * 20 * 304*288, dtype=np.float32)
do_inference(context, host_in, host_out)
print(f’Output: \n{host_out[:5000]}\n{host_out[-10:]}')
Environment
TensorRT Version: 8.0.1
GPU Type: AGX Xavier
Nvidia Driver Version:
CUDA Version: 10.2
CUDNN Version: 8.2
Operating System + Version: JetPack4.6
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered