Hi everyone, I don’t know where to post this, however… Currently, I am using yolov5 to inspect surface quality of products. Environment is Windows10Pro+python3.7-3.8 and YOLOv5. The model is custom-trained one based on yolov5s.pt. Model Summary: 213 layers, 7012822 parameters, 0 gradients, 15.8…

Hi, If you are using TensorRT, give Deepstream SDK a try. You can perform preprocessing/postprocessing in GPU memory itself. You’ll find a lot of repos related to yolov5 deepstream.

Hi rajupadhyay59, Thank you for your suggestion. I’ll check. According to Ultralytics’ report, inferece time using tensorrt is significantly shortened. Any comment or idea ? YOLOv5 🚀 v6.1-135-g7926afc torch 1.10.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB) Setup complete ✅ (8 CPUs, 51.0 GB R…

Yes, please convert your model to onnx first and then to tensorrt using trtexec or deepstream. Inferencing should be usually done in c++ when you are using tensorrt but you can also use deepstream sdk which uses “nvinfer element” to perform the inference in the gpu memory itself. Your actually yol…

Hi rajupadhyay59, As you can tell from my first post, engine is already FP16. It was converted by export.py written by Ultralytics, not by trtexec nor deepstream. They use ‘torch.onnx.export’ to create onnx-file, then ‘trt.OnnxParser’ to convert into engine-file. Does the way of conversion make …

Please convert it to onnx first and then use trtexec to create the engine. (I am unsure about the ‘trt.OnnxParser’ because I have no experience with it, maybe it works) I tried yolov8-seg on jetson today and even then it was giving me 30FPS on fp16. You can easily get inference for less than 5mili…

Want to halve inference time

AI & Data Science Deep Learning (Training & Inference) TensorRT

y.harada279 December 25, 2023, 6:12am 8

I found this. We need to run detection on Windows10 PC.
https://forums.developer.nvidia.com/t/does-deepstream-support-running-under-windows10-os/248951

Topic		Replies	Views
Python sample yolov3 app on tensorrt Jetson Xavier NX tensorrt , yolo , python	8	1851	August 3, 2020
TensorRT Inference is Slower Than Other Frameworks TensorRT	7	3903	December 9, 2019
Detectron2 model converted into Tensorrt taking large time TensorRT	1	607	September 20, 2022
Yolov4 TensorRT slower than Yolov4 darknet TensorRT	6	3641	September 1, 2020
Inference time? Jetson Xavier NX jetson-inference	2	547	August 3, 2021
Inference time speed up? TensorRT	1	526	August 3, 2021
Improve inference performances yolov5 Jetson Nano yolo , nano2gb	3	1694	June 2, 2022
Inference time is more for TensorRT engine than Pytorch model for Retinaface TensorRT tensorrt , pytorch	1	1117	June 4, 2021
Inference time increases in for loop TensorRT	2	467	February 6, 2023
YOLOV3 and YOLOV3-tiny Loading time in Deepstream DeepStream SDK	1	767	April 6, 2020

Want to halve inference time

Related topics