Hello @onurrcifcii,
It’s great to hear you managed to run the DeepStream example!
Results look great, that is awesome!
Now, regarding your question.
I believe you might be trying to tackle too many different problems at the same time.
I would suggest you divide and conquer.
Let’s start by looking at the problem set you have, in no specific order.
1. Train a custom model.
1.1 Get a dataset: You will require custom data to train your model. Believe it or not, this is generally the most expensive part of an AI project. If you already have a great dataset with proper annotations, you are up to an amazing start. If not, no worries, there are different ways you can get a dataset, such as using a free dataset, creating your own or even paying a third party company to do it for you.
1.2 Train your model: Once you have your own dataset, you can actually start training your model. To do this, you don’t necessarily need to use DeepStream. In fact, the usual procedure used in the market is for using a proper server to train on, you can either use your own or rent some online. You can train your model with whatever framework you feel most comfortable with, such as PyTorch for example. You can look around for different options, but for example, Ultralytics offer a tutorial on that.
1.3 Optimize your model: Once trained, your model will probably require a bunch of resources to run, given that it will likely be trained on a server. In order for your model to run on a Jetson Device making sure you get the best performance out if, you will need to optimize your model. For which you can use different techniques such as pruning or quantization. You could take a look at NVIDIA’s TensorRT Model Optimizer.
1.4 Integrate your model: Depending on your final model architecture, you might need to perform some integration steps in order to get it running on DeepStream. For instance, if your final layer outputs a different structure than the one on default Yolo, you might need to create a custom library for DeepStream to process the output results from your model. I would suggest you maintain Yolo’s output layer, so you can reuse as much as possible.
2. Create your custom App
2.1 Create a custom GStreamer pipeline: Create custom GStreamer pipeline that captures from the camera, then runs the images through inference and outputs the result into a simple stream so you can verify the system.
2.2 Embed the pipe into Python: Work on a simple python app that simply runs your pipeline. At this point, your only objective will be for the app to run the pipeline without you requiring to use get-launch.
2.3 Customize your app: Work on custom features. For instance, if you require the buffers to go through some pre or post processing on Python, this is your time to work on that. For instance, you could modify your pipe so you can tap into it and get buffers at whatever stage you need so you can interact with the buffers on python as you require.
I apologize if this is a bit too much into one comment, it is a complex but very interesting topic you are currently tackling. If I were you, I would choose either 1 or 2 and focus on just one for the moment. You can later work on the other or get some assistance.
Please do not hesitate to ask if you need further help.
regards,
Andrew
Embedded Software Engineer at ProventusNova
