Hi, there: I’m trying to use trtexec tool on Orin32 for a custom cnn model. If I let it run on one AI core, it works fine. However, if I try to run it in parallel, it will complain not enough GPU memory. We have an Orin64 board also, and it works just fine. Any way to make Orin32 works also? It…

There are many changes which I don’t fully understand. BTW, another vendor has two centernet onnx models, one resnet50 based and the other resnet101 based. Both can run well on Orin32 with gpu-only mode. Since there are private properties of another vendor, I can’t give to you either. Since our m…

I tried the approach. It still crashed . At the time of crash, the tegrastats shows the mem usage very low, only around 8GB. I set the workspace to 2600, 2700, 26000, 27000, all the same results. I rebooted the orin box after each try. BTW your unit is MB, so 26 GB should be 26000, not 2600. Anywa…

Hi, Sorry for the incorrect message about the amount. I have updated the comment above. Could you share the model with us so we can test it in our environment as well? Thanks.

Sorry, I can’t give out the model. It just has lots of cnn layers. For the last few layers, there are near 30 parallel conv layers, and then concat the results.

Hi, Just want to clarify the issue first. There is only one model but the model contains several layers that are expected to run parallel. Is that correct? Or you have multiple models that want to run concurrently? To get more information about the crash, could you add the --verbose for both Ori…

Just one model. We would like to run without specify the AI core to boot fps (or qps). I’am attaching 2 files: orin32_1dla.log (960.3 KB) orin64.log (10.3 MB) orin32_1dla.log: which succeeded with --useDLACore=0, low qps (only 23.5) orin64.log: which succeeded with the without --useDLACore, w…

Hi, Thanks for sharing. Based on your log, the memory configuration on both boards is set to ~30xxx MiB. So it’s expected that the behavior to be similar and the model should work on both boards. But it seems that the TensorRT version is 8.4.0 on the 64GB board. Do you set up the device with Je…

My issue is at the Orin32 board. Someone else owns the Orin64 board. I don’t know the details of setup. Could you please let me know how to fix orin32 setup? Orin32 TensorRt version dpkg -l | grep nvinfer ii libnvinfer-bin 8.4.1-1+cuda11.4 arm64…

[subl_model_orin (1)] I attached a subset of the model. Somehow it always fails at the red circle. If I remove both the red circle node and concat layer after, everything works. Or, if I keep the red circle, only deleting the concat after, everything still works. So odd, seems not related to mo…

Hi, Is it the layer listed below that name called node 772? 11/23/2022-17:12:13] [V] [TRT] Parsing node: node_of_772 [Conv] [11/23/2022-17:12:13] [V] [TRT] Searching for input: 771 [11/23/2022-17:12:13] [V] [TRT] Searching for input: hm_gcp.2.weight [11/23/2022-17:12:13] [V] [TRT] Searching for in…

Thanks. I can’t test for now, since I upgraded to TensorRT 8.5.1 and got new problem :( Still gpu memory though. Seems all my problems related to magic gpu mem. [image] Orin32, after upgrade TensorRT fron 8.4.1 to 8.5.1, trtexec no longer works General Topics and Other SDKs …

Trtexec model conversion crashed at insufficient gpu memory

Robotics & Edge Computing Jetson Systems Jetson Orin NX

firefly1688 December 9, 2022, 1:31am 24

Here is the details of the nearby nodes. The params are the same as other nodes in the model. Very odd. We have several similar models, all failed at the same place.

Topic		Replies	Views
Trtexec Internal Error (Symbolic relation a.z >= 0 is always false. ) TensorRT tensorrt	6	1144	December 5, 2022
Orin32, after upgrade TensorRT fron 8.4.1 to 8.5.1, trtexec no longer works General Topics & Other SDKs jetson-inference	1	839	December 9, 2022
Jetson Orin 32G. trtexec onnx->trt inside container Jetson AGX Orin tensorrt , containers	7	328	July 1, 2025
Using ONNX Runtime with TensorRT on Jetson Devices Jetson AGX Xavier tensorrt	5	1233	October 18, 2021
Skipping tactic 0x0000000000000000 due to Myelin error: Platform (Cuda) error Jetson Orin NX tensorrt	25	2742	January 25, 2023
Can't run nvcr.io/nvidia/l4t-tensorrt:r8.2.1-runtime on Orin AGX Jetson AGX Orin tensorrt	19	1377	May 13, 2022
Orin trtexec results in Segmentation Fault Jetson AGX Orin tensorrt , onnx	10	1907	March 3, 2023
TensorRT small model high RAM consumption during inference problem Jetson Orin Nano tensorrt , cuda , cudnn , yocto , jetson	10	390	November 7, 2024
Drive AGX Orin TensorRT inference failed DRIVE AGX Orin General driveos-dl	25	1373	September 14, 2023
Multiple executive warnings after switching tensorflow from 2.16.1 CPU to v60dp tensorflow==2.15.0+nv24.03 GPU version Jetson Orin Nano cudnn	8	2329	May 21, 2024

Trtexec model conversion crashed at insufficient gpu memory

Related topics