Hello Dustin,
I have passed over OpenCV error by removing ScopedRestrictedImport() as you suggested to me.
However, I am facing a new issue. I think it comes from the plugins. Please have a look on my Error log below:
make[1]: Entering directory '/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA'
[2022-11-07 10:08:15,252 main_v2.py:221 INFO] Detected system ID: KnownSystem.Orin
[2022-11-07 10:08:19,469 generate_engines.py:172 INFO] Building engines for resnet50 benchmark in Offline scenario...
[2022-11-07 10:08:19,548 ResNet50.py:36 INFO] Using workspace size: 1073741824
[11/07/2022-10:08:19] [TRT] [I] [MemUsageChange] Init CUDA: CPU +213, GPU +0, now: CPU 257, GPU 5728 (MiB)
[11/07/2022-10:08:22] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +351, GPU +333, now: CPU 627, GPU 6079 (MiB)
[2022-11-07 10:08:22,876 builder.py:107 INFO] Using DLA: Core 0
[2022-11-07 10:08:25,126 rn50_graphsurgeon.py:474 INFO] Renaming layers
[2022-11-07 10:08:25,127 rn50_graphsurgeon.py:485 INFO] Renaming tensors
[2022-11-07 10:08:25,127 rn50_graphsurgeon.py:834 INFO] Adding Squeeze
[2022-11-07 10:08:25,127 rn50_graphsurgeon.py:869 INFO] Adding Conv layer, instead of FC
[2022-11-07 10:08:25,130 rn50_graphsurgeon.py:890 INFO] Adding TopK layer
[2022-11-07 10:08:25,131 rn50_graphsurgeon.py:907 INFO] Removing obsolete layers
[2022-11-07 10:08:25,212 ResNet50.py:94 INFO] Unmarking output: topk_layer_output_value
[11/07/2022-10:08:25] [TRT] [W] DynamicRange(min: -128, max: 127). Dynamic range should be symmetric for better accuracy.
[2022-11-07 10:08:25,214 builder.py:177 INFO] Building ./build/engines/Orin/resnet50/Offline/resnet50-Offline-dla-b16-int8.lwis_k_99_MaxP.plan
[11/07/2022-10:08:25] [TRT] [W] Layer 'topk_layer': Unsupported on DLA. Switching this layer's device type to GPU.
[11/07/2022-10:08:25] [TRT] [I] Reading Calibration Cache for calibrator: EntropyCalibration2
[11/07/2022-10:08:25] [TRT] [I] Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
[11/07/2022-10:08:25] [TRT] [I] To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
[11/07/2022-10:08:29] [TRT] [I] ---------- Layers Running on DLA ----------
[11/07/2022-10:08:29] [TRT] [I] [DlaLayer] {ForeignNode[conv1...fc_replaced]}
[11/07/2022-10:08:29] [TRT] [I] ---------- Layers Running on GPU ----------
[11/07/2022-10:08:29] [TRT] [I] [GpuLayer] TOPK: topk_layer
[11/07/2022-10:08:31] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +535, GPU +302, now: CPU 1655, GPU 7065 (MiB)
[11/07/2022-10:08:31] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +85, GPU +83, now: CPU 1740, GPU 7148 (MiB)
[11/07/2022-10:08:31] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/07/2022-10:08:36] [TRT] [I] Detected 1 inputs and 1 output network tensors.
[11/07/2022-10:08:37] [TRT] [I] Total Host Persistent Memory: 848
[11/07/2022-10:08:37] [TRT] [I] Total Device Persistent Memory: 0
[11/07/2022-10:08:37] [TRT] [I] Total Scratch Memory: 1024
[11/07/2022-10:08:37] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 26 MiB, GPU 31 MiB
[11/07/2022-10:08:37] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 0.012416ms to assign 3 blocks to 3 nodes requiring 65536 bytes.
[11/07/2022-10:08:37] [TRT] [I] Total Activation Memory: 65536
[11/07/2022-10:08:37] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +26, GPU +4, now: CPU 26, GPU 4 (MiB)
[11/07/2022-10:08:37] [TRT] [I] The profiling verbosity was set to ProfilingVerbosity::kLAYER_NAMES_ONLY when the engine was built, so only the layer names will be returned. Rebuild the engine with ProfilingVerbosity::kDETAILED to get more verbose layer information.
[2022-11-07 10:08:37,338 builder.py:210 INFO] ========= TensorRT Engine Layer Information =========
[2022-11-07 10:08:37,339 builder.py:211 INFO] Layers:
{ForeignNode[conv1...fc_replaced]}
Reformatting CopyNode for Input Tensor 0 to topk_layer
input_tensor_0 finish
fc_replaced_out_0 finish
topk_layer
Bindings:
input_tensor_0
topk_layer_output_index
[11/07/2022-10:08:37] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[11/07/2022-10:08:37] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[2022-11-07 10:08:37,652 ResNet50.py:36 INFO] Using workspace size: 1073741824
[11/07/2022-10:08:37] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1453, GPU 7110 (MiB)
[2022-11-07 10:08:38,283 rn50_graphsurgeon.py:474 INFO] Renaming layers
[2022-11-07 10:08:38,283 rn50_graphsurgeon.py:485 INFO] Renaming tensors
[2022-11-07 10:08:38,284 rn50_graphsurgeon.py:834 INFO] Adding Squeeze
[2022-11-07 10:08:38,284 rn50_graphsurgeon.py:869 INFO] Adding Conv layer, instead of FC
[2022-11-07 10:08:38,286 rn50_graphsurgeon.py:890 INFO] Adding TopK layer
[2022-11-07 10:08:38,287 rn50_graphsurgeon.py:907 INFO] Removing obsolete layers
[2022-11-07 10:08:38,289 rn50_graphsurgeon.py:580 INFO] Fusing ops in res2_mega
[2022-11-07 10:08:38,292 rn50_graphsurgeon.py:693 INFO] Plugin RES2_FULL_FUSION successful
[2022-11-07 10:08:38,292 rn50_graphsurgeon.py:499 INFO] Replacing all branch2c beta=1 conv with smallk kernel.
[2022-11-07 10:08:38,292 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,293 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,293 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,293 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3d_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,293 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,293 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4d_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4e_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4f_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,294 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,295 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:08:38,297 rn50_graphsurgeon.py:573 INFO] Plugin SmallTileGEMM_TRT fused successful for res3/4/5 branch2c
[11/07/2022-10:08:38] [TRT] [I] No importer registered for op: RnRes2FullFusion_TRT. Attempting to import as plugin.
[11/07/2022-10:08:38] [TRT] [I] Searching for plugin: RnRes2FullFusion_TRT, plugin_version: 1, plugin_namespace:
[11/07/2022-10:08:38] [TRT] [I] Successfully created plugin: RnRes2FullFusion_TRT
[11/07/2022-10:08:38] [TRT] [I] No importer registered for op: SmallTileGEMM_TRT. Attempting to import as plugin.
[11/07/2022-10:08:38] [TRT] [I] Searching for plugin: SmallTileGEMM_TRT, plugin_version: 1, plugin_namespace:
[11/07/2022-10:08:38] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBias not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:08:38] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasRelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:08:38] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasGelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:08:38] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasBeta not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:08:38] [TRT] [F] Validation failed: false
plugin/smallTileGEMMPlugin/smallTileGEMMPlugin.cpp:520
[11/07/2022-10:08:38] [TRT] [E] std::exception
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/base.py", line 185, in subprocess_target
return self.action_handler.handle()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/generate_engines.py", line 175, in handle
total_engine_build_time += self.build_engine(job)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/generate_engines.py", line 166, in build_engine
builder.build_engines()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/common/builder.py", line 170, in build_engines
self.initialize()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/resnet50/tensorrt/ResNet50.py", line 87, in initialize
raise RuntimeError(f"ResNet50 onnx model processing failed! Error: {err_desc}")
RuntimeError: ResNet50 onnx model processing failed! Error: Assertion failed: plugin && "Could not create plugin"
[2022-11-07 10:08:39,914 generate_engines.py:172 INFO] Building engines for resnet50 benchmark in Offline scenario...
[2022-11-07 10:08:39,972 ResNet50.py:36 INFO] Using workspace size: 1073741824
[11/07/2022-10:08:40] [TRT] [I] [MemUsageChange] Init CUDA: CPU +213, GPU +0, now: CPU 257, GPU 5821 (MiB)
[11/07/2022-10:08:42] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +351, GPU +337, now: CPU 627, GPU 6180 (MiB)
[2022-11-07 10:08:42,798 builder.py:107 INFO] Using DLA: Core 0
[2022-11-07 10:08:43,389 rn50_graphsurgeon.py:474 INFO] Renaming layers
[2022-11-07 10:08:43,390 rn50_graphsurgeon.py:485 INFO] Renaming tensors
[2022-11-07 10:08:43,390 rn50_graphsurgeon.py:834 INFO] Adding Squeeze
[2022-11-07 10:08:43,390 rn50_graphsurgeon.py:869 INFO] Adding Conv layer, instead of FC
[2022-11-07 10:08:43,393 rn50_graphsurgeon.py:890 INFO] Adding TopK layer
[2022-11-07 10:08:43,393 rn50_graphsurgeon.py:907 INFO] Removing obsolete layers
[2022-11-07 10:08:43,472 ResNet50.py:94 INFO] Unmarking output: topk_layer_output_value
[11/07/2022-10:08:43] [TRT] [W] DynamicRange(min: -128, max: 127). Dynamic range should be symmetric for better accuracy.
[2022-11-07 10:08:43,473 builder.py:177 INFO] Building ./build/engines/Orin/resnet50/Offline/resnet50-Offline-dla-b16-int8.lwis_k_99_MaxP.plan
[11/07/2022-10:08:43] [TRT] [W] Layer 'topk_layer': Unsupported on DLA. Switching this layer's device type to GPU.
[11/07/2022-10:08:43] [TRT] [I] Reading Calibration Cache for calibrator: EntropyCalibration2
[11/07/2022-10:08:43] [TRT] [I] Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
[11/07/2022-10:08:43] [TRT] [I] To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
[11/07/2022-10:08:48] [TRT] [I] ---------- Layers Running on DLA ----------
[11/07/2022-10:08:48] [TRT] [I] [DlaLayer] {ForeignNode[conv1...fc_replaced]}
[11/07/2022-10:08:48] [TRT] [I] ---------- Layers Running on GPU ----------
[11/07/2022-10:08:48] [TRT] [I] [GpuLayer] TOPK: topk_layer
[11/07/2022-10:08:49] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +535, GPU +259, now: CPU 1655, GPU 7185 (MiB)
[11/07/2022-10:08:50] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +85, GPU +88, now: CPU 1740, GPU 7273 (MiB)
[11/07/2022-10:08:50] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/07/2022-10:08:57] [TRT] [I] Detected 1 inputs and 1 output network tensors.
[11/07/2022-10:08:57] [TRT] [I] Total Host Persistent Memory: 848
[11/07/2022-10:08:57] [TRT] [I] Total Device Persistent Memory: 0
[11/07/2022-10:08:57] [TRT] [I] Total Scratch Memory: 1024
[11/07/2022-10:08:57] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 26 MiB, GPU 31 MiB
[11/07/2022-10:08:57] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 0.010496ms to assign 3 blocks to 3 nodes requiring 65536 bytes.
[11/07/2022-10:08:57] [TRT] [I] Total Activation Memory: 65536
[11/07/2022-10:08:57] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +26, GPU +4, now: CPU 26, GPU 4 (MiB)
[11/07/2022-10:08:57] [TRT] [I] The profiling verbosity was set to ProfilingVerbosity::kLAYER_NAMES_ONLY when the engine was built, so only the layer names will be returned. Rebuild the engine with ProfilingVerbosity::kDETAILED to get more verbose layer information.
[2022-11-07 10:08:57,709 builder.py:210 INFO] ========= TensorRT Engine Layer Information =========
[2022-11-07 10:08:57,709 builder.py:211 INFO] Layers:
{ForeignNode[conv1...fc_replaced]}
Reformatting CopyNode for Input Tensor 0 to topk_layer
input_tensor_0 finish
fc_replaced_out_0 finish
topk_layer
Bindings:
input_tensor_0
topk_layer_output_index
[11/07/2022-10:08:57] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[11/07/2022-10:08:57] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[2022-11-07 10:08:59,325 ResNet50.py:36 INFO] Using workspace size: 1073741824
[11/07/2022-10:08:59] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1453, GPU 7323 (MiB)
[2022-11-07 10:09:03,892 rn50_graphsurgeon.py:474 INFO] Renaming layers
[2022-11-07 10:09:03,893 rn50_graphsurgeon.py:485 INFO] Renaming tensors
[2022-11-07 10:09:03,893 rn50_graphsurgeon.py:834 INFO] Adding Squeeze
[2022-11-07 10:09:03,893 rn50_graphsurgeon.py:869 INFO] Adding Conv layer, instead of FC
[2022-11-07 10:09:03,896 rn50_graphsurgeon.py:890 INFO] Adding TopK layer
[2022-11-07 10:09:03,896 rn50_graphsurgeon.py:907 INFO] Removing obsolete layers
[2022-11-07 10:09:03,899 rn50_graphsurgeon.py:580 INFO] Fusing ops in res2_mega
[2022-11-07 10:09:03,901 rn50_graphsurgeon.py:693 INFO] Plugin RES2_FULL_FUSION successful
[2022-11-07 10:09:03,901 rn50_graphsurgeon.py:499 INFO] Replacing all branch2c beta=1 conv with smallk kernel.
[2022-11-07 10:09:03,902 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,902 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,902 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,902 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res3d_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,902 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,903 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,903 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,903 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4d_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,903 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4e_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,903 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res4f_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,904 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5a_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,904 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5b_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,904 rn50_graphsurgeon.py:566 INFO] Fusing SmallTileGEMM_TRT_res5c_branch2c_conv_residual_relu with smallk...
[2022-11-07 10:09:03,906 rn50_graphsurgeon.py:573 INFO] Plugin SmallTileGEMM_TRT fused successful for res3/4/5 branch2c
[11/07/2022-10:09:03] [TRT] [I] No importer registered for op: RnRes2FullFusion_TRT. Attempting to import as plugin.
[11/07/2022-10:09:03] [TRT] [I] Searching for plugin: RnRes2FullFusion_TRT, plugin_version: 1, plugin_namespace:
[11/07/2022-10:09:03] [TRT] [I] Successfully created plugin: RnRes2FullFusion_TRT
[11/07/2022-10:09:03] [TRT] [I] No importer registered for op: SmallTileGEMM_TRT. Attempting to import as plugin.
[11/07/2022-10:09:03] [TRT] [I] Searching for plugin: SmallTileGEMM_TRT, plugin_version: 1, plugin_namespace:
[11/07/2022-10:09:03] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBias not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:09:03] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasRelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:09:03] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasGelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:09:03] [TRT] [W] builtin_op_importers.cpp:4714: Attribute epilogueScaleBiasBeta not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[11/07/2022-10:09:03] [TRT] [F] Validation failed: false
plugin/smallTileGEMMPlugin/smallTileGEMMPlugin.cpp:520
[11/07/2022-10:09:03] [TRT] [E] std::exception
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/base.py", line 185, in subprocess_target
return self.action_handler.handle()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/generate_engines.py", line 175, in handle
total_engine_build_time += self.build_engine(job)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/generate_engines.py", line 166, in build_engine
builder.build_engines()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/common/builder.py", line 170, in build_engines
self.initialize()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/resnet50/tensorrt/ResNet50.py", line 87, in initialize
raise RuntimeError(f"ResNet50 onnx model processing failed! Error: {err_desc}")
RuntimeError: ResNet50 onnx model processing failed! Error: Assertion failed: plugin && "Could not create plugin"
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/main_v2.py", line 223, in <module>
main(main_args, DETECTED_SYSTEM)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/main_v2.py", line 147, in main
dispatch_action(main_args, config_dict, workload_setting)
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/main_v2.py", line 194, in dispatch_action
handler.run()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/base.py", line 79, in run
self.handle_failure()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/base.py", line 182, in handle_failure
self.action_handler.handle_failure()
File "/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA/code/actionhandler/generate_engines.py", line 183, in handle_failure
raise RuntimeError("Building engines failed!")
RuntimeError: Building engines failed!
make[1]: *** [Makefile:694: generate_engines] Error 1
make[1]: Leaving directory '/MLPerfv2.1/inference_results_v2.1/closed/NVIDIA'
make: *** [Makefile:688: run] Error 2
The command was :
$ make run RUN_ARGS="--benchmarks=resnet50 --scenarios=Offline --test_mode=AccuracyOnly"
I tried to run this one as well, the issue is pretty much the same as the first one.
$ make run RUN_ARGS="--benchmarks=resnet50 --scenarios=Offline --test_mode=PerformanceOnly"
Thank you in advance.
Best regards
Harry