I have made a wrapper to the deepstream trt-yolo program. It was not easy, but its done.
Inference speed on Nano 10w (not MAXN) is 85ms/image (including pre-processing and NMS - not like the NVIDIA benchmarks :) ), which is FAR faster then anything I have tried.
Also load time is very fast after the first engine compilation.
The code is a bit rough and still needs a lot of attention but I would be grateful if anyone can try and follow the installation because, sadly, I ran out of memory cards and don’t want to erase this one.
The github is at https://github.com/mosheliv/tensortrt-yolo-python-api . Feel free to contribute to it - I still need to implement wrapper for Yolov3 (shouldn’t be hard) and add support for better image resizing logic (currently 416 is hardcoded) and error handling.
Any comments are welcome. It has been a while since I used c++ (we are talking decades here) and this wrapper was tricky.