Should I Use Jetson Multimedia API or DeepStreamSDK or custom gstreamer plugins?

[user144103]

Hello. I am quite new to gstreamer and Jetson Nano. I am developing an application using a CSI camera with Jetson Nano. The application pipeline has these components:

1. Stream video from CSI camera.
2. Dynamic cropping of video (via buttons wired to GPIO ports). I have read that Nvidia’s plugin nvvidconv does not support dynamic change of its properties.
3. Color space changing (nvvidconv does the job here).
4. Use CNNs on the video frames for object detection.
5. Drawing geometric shapes (such as rectangular bounding boxes) on the video

Some of the operations like dynamic cropping can be done with gstreamer plugins such as videocrop. However that requires the data to be transferred from NVVM to non-NVVM memory. The camera outputs 1920x1080 frames, so doing operations using gstreamer plugins results in considerable lag. I am looking for a way to use the Jetson’s hardware acceleration capabilities while doing all of the above operations.

So my question is which of the following options should I choose:

1. Jetson Multimedia API
2. DeepStreamSDK
3. Write my own gstreamer plugins in C/C++/Python

If possible, I would like to use Python for development. Any help on this would be highly appreciated.

[DaneLLL]

Hi,
Using jestson_multimedia_api is more flexible in developing advancing use-case but need to implement the functions that are already in gstreamer plugins.

The nvvidconv plugin is open source so you may consider to implement dynamic cropping in nvvidconv and rebuild/replace the plugin. If the output resolution is fixed after dynamic cropping, it should be possible. If resolution changes dynamically, it is better to use jestson_multimedia_api.

[user144103]

Thank you for the advice.

Modifying the nvvidconv code to enable dynamic cropping seems to be quicker fix than learning/implementing the jetson_multimedia_api. As a temporary solution I am using videocrop plugin for dynamic cropping.

@user144103

Drawing geometric shapes (such as rectangular bounding boxes) on the video

For this I am using rsvgoverlay. This allows me to programmatically change the size/shape of the geometry using simple Python code. Also I don’t have to deal with modifying the raw buffers, which can cause lag if done in Python.