Dynamic shapes not dynamic at all

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

There seems to be a bug where a model crashes if launched with a different shape than the one provided in the opt field of the optimization profile. It is a stereo vision model whose inputs are three images, two in color and one in grayscale. When compiled with the following optimization profile:

SetTrtProfileMinShapes("img1:1x3x512x512,img2:1x3x512x512,disp_init:1x512x512");
SetTrtProfileOptShapes("img1:1x3x1024x896,img2:1x3x1024x896,disp_init:1x1024x896");
SetTrtProfileMaxShapes("img1:1x3x2048x2048,img2:1x3x2048x2048,disp_init:1x2048x2048");

And run with images of shape 1024x768 it crashes with error:

Non-zero status code returned while running TRTKernel_graph_main_graph_7509437748431248464_0 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_main_graph_7509437748431248464_0_0' Status Message: TensorRT EP execution context enqueue failed.

On the other hand, if it is compiled with:

SetTrtProfileMinShapes("img1:1x3x512x512,img2:1x3x512x512,disp_init:1x512x512");
SetTrtProfileOptShapes("img1:1x3x1024x768,img2:1x3x1024x768,disp_init:1x1024x768");
SetTrtProfileMaxShapes("img1:1x3x2048x2048,img2:1x3x2048x2048,disp_init:1x2048x2048");

It now crashes when run with images of shape 1024x896. This does not make sense. 768 and 896 are both between the minimum and the maximum, why do I need to specify the exact value in the opt field? Even if I set a higher value to the width like 1024 it crashes. It needs to be the exact value that is used later.

What’s even more confusing is that it only crashes on Blackwell GPUs. The same code, model and data does not crash in a A5000 for this scenario. However, in 5080 and 5060 Ti it crashes. If there is something else we need to configure, please let us know. We just want to have real dynamic shapes without crashing. If it causes a recompilation is fine, but a crash is inadmissible.

Environment

TensorRT Version: 10.13.0.35
GPU Type: Blackwell Series 50
Nvidia Driver Version: 595.97
CUDA Version: 12.4
CUDNN Version: 8.9.7
Operating System + Version: Windows 10 / 11
ONNXRuntime: 1.18.0

Relevant Files

Model onnx file:

Steps To Reproduce

The full codebase is very convoluted so I won’t share it. The idea is to load the model using onnxruntime in C++ with TensorRT backend. For the images you can just create zero images of the specified shapes and the crash will happen too, it is independent of the data. If you cannot reproduce, then that counts as a solution so please just provide the script to not reproduce so that we can compare on our side and apply the changes to make it work.

2

Hi @jose.perez.cano, Thanks for the detailed write-up and for narrowing it down this far. The pattern points away from your profile setup: an engine that runs only at the exact opt shape, faults everywhere else inside [min, max], is clean on the A5000, and crashes on the 5080 and 5060 Ti is a hardware-specific codegen signature, not a misconfigured optimization profile.

I pulled your waft_refine.onnx into Netron and the model explains it. It is WAFT-Stereo refine, with dynamic H/W on all three inputs, and it is packed with ops that TensorRT’s Myelin compiler fuses and locks to one shape: 55 InstanceNormalization nodes, a ViT attention stack (LayerNormalization + MatMul + Softmax), 24 Resize layers in the upsampler, the grid_sample warp, and Range/Expand pairs that build the sampling grid at runtime. When you pass min/opt/max, Myelin tunes those kernels for the opt dimensions. On most targets the path for other in-range shapes is safe. On consumer Blackwell (sm_120) in TRT 10.13, one of the fused kernels indexes out of bounds at non-opt shapes, and ORT surfaces that as TensorRT EP execution context enqueue failed (the underlying enqueueV3 returns an error). The A5000 stays clean because Ampere picks a different kernel path.

This matches a Myelin dynamic-shape regression on the InstanceNormalization fusion reported here a couple of months ago, also “only the opt shape works”:

Here is how to unblock today and confirm the cause.

1. Isolate core TensorRT from the ORT integration

Build the engine directly with trtexec, then run it at a non-opt shape. If it faults the same way, the bug is in TensorRT’s Myelin path rather than the ORT TRT EP, and you have a clean repro to file.

trtexec --onnx=waft_refine.onnx --saveEngine=waft.plan ^
  --minShapes=img1:1x3x512x512,img2:1x3x512x512,disp_init:1x512x512 ^
  --optShapes=img1:1x3x1024x896,img2:1x3x1024x896,disp_init:1x1024x896 ^
  --maxShapes=img1:1x3x2048x2048,img2:1x3x2048x2048,disp_init:1x2048x2048 ^
  --verbose --profilingVerbosity=detailed

trtexec --loadEngine=waft.plan --shapes=img1:1x3x1024x768,img2:1x3x1024x768,disp_init:1x1024x768

(On Windows the ^ is the line-continuation character; you can also put it on one line.)

2. Pin each ORT TensorRT EP profile to a static shape (works now)

You said a per-resolution recompile is acceptable, which is the right trade. Set min == opt == max to the resolution you are about to run, so the engine is static and the dynamic-shape Myelin path never fires. Turn on the engine cache so each resolution builds once.

provider_options = {
    "trt_engine_cache_enable": True,
    "trt_engine_cache_path": "./trt_engines",
    "trt_profile_min_shapes": "img1:1x3x1024x768,img2:1x3x1024x768,disp_init:1x1024x768",
    "trt_profile_opt_shapes": "img1:1x3x1024x768,img2:1x3x1024x768,disp_init:1x1024x768",
    "trt_profile_max_shapes": "img1:1x3x1024x768,img2:1x3x1024x768,disp_init:1x1024x768",
}

You keep multi-resolution support (one cached engine per resolution you feed it), without the crash.

The first run at each new resolution pays a one-time build, so warm the cache for your common sizes first.

3. Move to a newer TensorRT and align CUDA to Blackwell

TRT 10.13 is early for consumer Blackwell, and sm_120 wants a CUDA 12.8+ toolchain while your stack reports 12.4. Rebuilding ORT against a newer TensorRT on CUDA 12.8+ is the most likely permanent fix. The per-architecture CUDA floor and the engine-portability rules are in the Support Matrix.

4. If you are still stuck, drop the builder optimization level

A lower level sometimes avoids the fused tactic that miscompiles. Set trt_builder_optimization_level to 2 or 3 in the EP options. Lower confidence, but it costs one rebuild to check.

Quickest path today is #2. In parallel, if #1 shows trtexec faulting the same way, I will file this with the compiler team as a Blackwell sm_120 Myelin dynamic-shape issue and post a tracking ID here. I will also try to reproduce it on a 50-series card over the next few days.

One quick data point: do you have trt_fp16_enable on in the EP options? On the related InstanceNorm regression the fault was FP16-specific, so whether the non-opt shapes pass in plain FP32 tells us fast if this is the same path.

Thanks,
Atharva