# Trt model usage code in python

**URL:** <https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239>\
**Category:** Jetson Orin Nano\
**Tags:** tensorrt\
**Created:** [January 8, 2025, 11:10am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239 "2025-01-08T11:10:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![KY\_Jetsy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ky_jetsy/32/438083_2.png) [@KY\_Jetsy](https://forums.developer.nvidia.com/u/KY_Jetsy)\
**Post date:** [January 8, 2025, 11:10am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/1 "2025-01-08T11:10:23Z")

</div>

Hi,

I had a .pt model from custom yolov5. i had converted it to .onnx later after some days. now i tried to convert it to .trt model using this command:

**trtexec --onnx=/home/nvidia/PycharmProjects/jetson-linuxsphu/nhb.onnx --saveEngine=/home/nvidia/PycharmProjects/jetson-linuxsphu/nhb.trt --fp16**

the conversion was succesfull. im sharing the log file.  
[trt\_convert.txt](https://forums.developer.nvidia.com/uploads/short-url/bqXRWeGFxLD5teNtVpSj0JE4w74.txt) (31.8 KB)

But i dont know how to use it in python code for inferencing. Can someone help me with a sample code?

MY DEVICE SPECS:

 ![Screenshot from 2025-01-08 16-38-31](https://global.discourse-cdn.com/nvidia/original/4X/4/7/f/47f50c142cdf5c28b53d8ca47923986a5e03175f.png)

Thanks

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [January 9, 2025, 4:50am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/3 "2025-01-09T04:50:16Z")

</div>

Hi,

Please find a related sample below:

> <https://github.com/NVIDIA/TensorRT/blob/release/8.6/samples/python/yolov3_onnx/onnx_to_tensorrt.py>

Thanks.

---

<div class="post-metadata">

**Author:** ![KY\_Jetsy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ky_jetsy/32/438083_2.png) [@KY\_Jetsy](https://forums.developer.nvidia.com/u/KY_Jetsy)\
**Post date:** [January 9, 2025, 8:31am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/4 "2025-01-09T08:31:13Z")

</div>

Hi,

I tried it. but getting this error:  
/home/nvidia/PycharmProjects/jetson-linuxsphu/venv/bin/python /home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py  
Traceback (most recent call last):  
File “/home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py”, line 12, in   
import common  
File “/home/nvidia/PycharmProjects/jetson-linuxsphu/common.py”, line 25, in   
from cuda import cuda, cudart  
ModuleNotFoundError: No module named ‘cuda’

Process finished with exit code 1

In the onnx\_to\_tensorrt.py file, i had specified my .trt file and an image file for inference.

Attaching img file where the error is pointing to.

 ![Screenshot from 2025-01-09 13-59-51](https://global.discourse-cdn.com/nvidia/original/4X/6/5/1/6519e27dad4a72aaac09e21a2cf1912eb0ea001b.png)  
The above img file is showing the code part from common.py from same github repo

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [January 13, 2025, 7:03am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/5 "2025-01-13T07:03:03Z")

</div>

Hi,

Could you try if you can install it with the below command:

```auto
$ pip3 install pycuda --user

```

Thanks.

---

<div class="post-metadata">

**Author:** ![KY\_Jetsy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ky_jetsy/32/438083_2.png) [@KY\_Jetsy](https://forums.developer.nvidia.com/u/KY_Jetsy)\
**Post date:** [January 13, 2025, 8:32am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/6 "2025-01-13T08:32:17Z")

</div>

> [@AastaLLL](#):
>
> `pip3 install pycuda --user`

nvidia@tegra-ubuntu:~$ pip3 install pycuda --user  
Requirement already satisfied: pycuda in ./.local/lib/python3.8/site-packages (2024.1)  
Requirement already satisfied: pytools\>=2011.2 in ./.local/lib/python3.8/site-packages (from pycuda) (2023.1.1)  
Requirement already satisfied: appdirs\>=1.4.0 in ./.local/lib/python3.8/site-packages (from pycuda) (1.4.4)  
Requirement already satisfied: mako in /usr/lib/python3/dist-packages (from pycuda) (1.1.0)  
Requirement already satisfied: platformdirs\>=2.2.0 in ./.local/lib/python3.8/site-packages (from pytools\>=2011.2-\>pycuda) (4.1.0)  
Requirement already satisfied: typing-extensions\>=4.0 in ./.local/lib/python3.8/site-packages (from pytools\>=2011.2-\>pycuda) (4.12.2)

[notice] A new release of pip is available: 23.3.2 → 24.3.1  
[notice] To update, run: python3 -m pip install --upgrade pip

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [January 15, 2025, 5:37am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/7 "2025-01-15T05:37:03Z")

</div>

Hi,

Sorry, it should be cuda-python.

```auto
$ sudo apt install python3-pip
$ pip3 install cuda-python

```

```auto
$ python3
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cuda import cuda, cudart
>>>

```

Thanks.

---

<div class="post-metadata">

**Author:** ![KY\_Jetsy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ky_jetsy/32/438083_2.png) [@KY\_Jetsy](https://forums.developer.nvidia.com/u/KY_Jetsy)\
**Post date:** [January 15, 2025, 5:54am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/8 "2025-01-15T05:54:09Z")

</div>

Hi,  
I used the same commands and the cuda-python was installed successfully. but when i run the code now it throws this error:

\*/home/nvidia/PycharmProjects/jetson-linuxsphu/venv/bin/python /home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py \*  
_Reading engine from file nhb.trt_  
_Running inference on image 4.jpg…_  
**Traceback (most recent call last):**

- File “/home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py”, line 160, in \*
- main()\*
- File “/home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py”, line 125, in main\*
- trt\_outputs = [output.reshape(shape) for output, shape in zip(trt\_outputs, output\_shapes)]\*
- File “/home/nvidia/PycharmProjects/jetson-linuxsphu/pspp.py”, line 125, in \*
- trt\_outputs = [output.reshape(shape) for output, shape in zip(trt\_outputs, output\_shapes)]\*  
**ValueError: cannot reshape array of size 176400 into shape (1,255,19,19)**

_Process finished with exit code 1_

I think this is because of model mismatch. The one used in the example was yolov3, but i have a custom yolov5 with 2 classes trained to detect. Is there any way to solve this? Please help.

Thanks

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [January 16, 2025, 7:53am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/9 "2025-01-16T07:53:07Z")

</div>

Hi,

The sample is open-sourced.  
You can modify the output based on your model accordingly.

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [February 26, 2025, 12:53am UTC](https://forums.developer.nvidia.com/t/trt-model-usage-code-in-python/319239/11 "2025-02-26T00:53:16Z")

</div>

Closed on February 26, 2025, 12:53am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
