MLPerf inference 2.1 results on AGX Orin

Hi, I’m trying to reproduce MLPerf results 2.1 on AGX Orin. I’ve been following the steps from here: inference_results_v2.1/closed/NVIDIA at master · mlcommons/inference_results_v2.1 · GitHub

I am able to run Resnet in the SingleStream scenario, but I see the below error when I try the Offline scenario. Please advise.

[10/05/2022-10:45:55] [TRT] [I] No importer registered for op: SmallTileGEMM_TRT. Attempting to import as plugin.
[10/05/2022-10:45:55] [TRT] [I] Searching for plugin: SmallTileGEMM_TRT, plugin_version: 1, plugin_namespace:
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBias not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasRelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasGelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasBeta not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [F] Assertion failed: false
smallTileGEMMPlugin.cu:590
Aborting…

Hi @virtual.ramblings,

These warning messages from TRT are expected.

[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBias not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasRelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasGelu not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[10/05/2022-10:45:55] [TRT] [W] builtin_op_importers.cpp:4765: Attribute epilogueScaleBiasBeta not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.</i>

The assertion failure is likely due to a wrong version of TensorRT being used - for MLPerf we have a different setup flow, details can be found at:

Thanks! The only way to upgrade TensorRT is to use this Developer Preview Jetpack?

Yes, the only way to get the version of TensorRT that the MLPerf Inference v2.1 build expects is to use that 22.08 Developer Preview.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.