Error Code 2: Internal Error (Engine generation failed because all backend strategies failed.)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
, ,
1

Hi,
I’m trying to convert an model using trtexec on Orin Nano 8GB. The conversion succeeds with an input resolution of 320×576, but fails when the resolution exceeds this size. Below is the log:

[01/01/1970-10:14:52] [W] [TRT] Tactic Device request: 144MB Available: 136MB. Device memory is insufficient to use tactic.
[01/01/1970-10:14:52] [W] [TRT] UNSUPPORTED_STATE: Skipping tactic 51 due to insufficient memory on requested size of 150994944 detected for tactic 0xa40cb43c296a36a8.
[01/01/1970-10:14:52] [W] [TRT] Tactic Device request: 144MB Available: 136MB. Device memory is insufficient to use tactic.
[01/01/1970-10:14:52] [W] [TRT] UNSUPPORTED_STATE: Skipping tactic 52 due to insufficient memory on requested size of 150994944 detected for tactic 0x5d067c18f40c23e3.
[01/01/1970-10:14:52] [W] [TRT] Tactic Device request: 144MB Available: 136MB. Device memory is insufficient to use tactic.
[01/01/1970-10:14:52] [W] [TRT] UNSUPPORTED_STATE: Skipping tactic 53 due to insufficient memory on requested size of 150994944 detected for tactic 0x263a38afd75e3a43.
[01/01/1970-10:14:52] [V] [TRT] /feature/dino/output_conv1/Conv (CaskConvolution[0x80000009]) profiling completed in 0.27668 seconds. Fastest Tactic: 0xd15ea5edd15ea5ed Time: inf
[01/01/1970-10:14:52] [V] [TRT] Skipping CaskFlattenConvolution: No valid tactics for /feature/dino/output_conv1/Conv
[01/01/1970-10:14:53] [W] [TRT] Engine generation failed with backend strategy 2.
Error message: [optimizer.cpp::computeCosts::4151] Error Code 10: Internal Error (Could not find any implementation for node /feature/dino/output_conv1/Conv.).
Skipping this backend strategy.
[01/01/1970-10:14:53] [E] Error[2]: [engineBuilder.cpp::makeEngineFromSubGraph::1879] Error Code 2: Internal Error (Engine generation failed because all backend strategies failed.)
[01/01/1970-10:14:53] [E] Engine could not be created from network
[01/01/1970-10:14:53] [E] Building engine failed
[01/01/1970-10:14:53] [E] Failed to create engine from model or file.
[01/01/1970-10:14:53] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v100300] # ./trtexec --onnx=/home/nvidia/fs-352-640.onnx --saveEngine=/home/nvidia/test.engine --fp16 --verbose --builderOptimizationLevel=4

Thanks in advance for your help.
Best regards

3

Hi,

Please try --builderOptimizationLevel=5 to see if it can work.
If the flag doesn’t work, could you share the model with us so we can check it further?

Thanks.

5

Hi,

I tried and it didn’t work.
Here is the model:

Thanks.

6

Hi,

This is a large model.
Could you run trtexec with --verbose to see if you are running out of memory?

Thanks.

7

Hi,

I think I’m running out of memory. Before the error occurred, it printed a long list of Skipping tactic xx due to insufficient memory on requested size of...
Does this mean the model can’t be converted? Is there any workaround for this?

Thanks.

8

Hi,

The model is too large to run on the Orin Nano 8GB.

We got this error when deploy the model with INT8 mode:

$ /usr/src/tensorrt/bin/trtexec --onnx=fs-352-640.onnx --int8
...
[04/10/2025-07:11:40] [W] [TRT] Tactic Device request: 1218MB Available: 716MB. Device memory is insufficient to use tactic.
[04/10/2025-07:11:40] [W] [TRT] UNSUPPORTED_STATE: Skipping tactic 0 due to insufficient memory on requested size of 1277224960 detected for tactic 0x0000000000000000.
[04/10/2025-07:11:41] [E] Error[10]: IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/Concat_333_output_0.../Concat_335]}.)
[04/10/2025-07:11:41] [E] Engine could not be created from network
[04/10/2025-07:11:41] [E] Building engine failed
[04/10/2025-07:11:41] [E] Failed to create engine from model or file.
[04/10/2025-07:11:41] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v100300] # /usr/src/tensorrt/bin/trtexec --onnx=fs-352-640.onnx --int8

But pass on another Orin NX 16GB mode with the same TensorRT setting:

$ /usr/src/tensorrt/bin/trtexec --onnx=fs-352-640.onnx --int8
...
[04/10/2025-08:24:52] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +4, GPU +1352, now: CPU 6, GPU 2208 (MiB)
[04/10/2025-08:24:52] [I] Setting persistentCacheLimit to 0 bytes.
[04/10/2025-08:24:52] [I] Created execution context with device memory size: 1347.65 MiB
[04/10/2025-08:24:52] [I] Using random values for input left
[04/10/2025-08:24:52] [I] Input binding for left with dimensions 1x3x640x352 is created.
[04/10/2025-08:24:52] [I] Using random values for input right
[04/10/2025-08:24:52] [I] Input binding for right with dimensions 1x3x640x352 is created.
[04/10/2025-08:24:52] [I] Output binding for output with dimensions 1x1x640x352 is created.
[04/10/2025-08:24:52] [I] Starting inference
[04/10/2025-08:25:08] [I] Warmup completed 1 queries over 200 ms
[04/10/2025-08:25:08] [I] Timing trace has 10 queries over 14.3177 s
[04/10/2025-08:25:08] [I] 
...
&&&& PASSED TensorRT.trtexec [TensorRT v100300] # /usr/src/tensorrt/bin/trtexec --onnx=fs-352-640.onnx --int8

Thanks.

9

Thanks anyway for trying to help! I really appreciate your time and input.