[Deepstream] How can I do my own image processing in C++?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

The following pipeline works fine.

gst-launch-1.0 -ev \
nvv4l2camerasrc device=/dev/video0 ! \
"video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1920, height=(int)1080, interlace-mode=progressive, framerate=(fraction)60/1" ! \
nvvideoconvert ! \
nv3dsink

Target

I want to receive the image in GPU memory in a C++ program before nv3dsink, process the image in some way, return it to the pipeline, and display it in nv3dsink.

Question

If I receive image in CPU memory in a C++ program, I can use appsink and appsrc, but what function should I use to receive image in GPU memory?
I often see nvdsosd in deepstream sample programs, is it possible by using it? It is used for drawing bounding boxes and text, rather than image processing in my shallow understanding of this function.

Please note that I am a beginner with 1 month experience with Gstreamer and 1 week experience with Deepstream.

• Hardware Platform (Jetson / GPU): Jetson Orin AGX 32GB Developer kit
• DeepStream Version: 6.1
• JetPack Version (valid for Jetson only): 5.02 (L4T: 35.1)
• TensorRT Version: 8.4.1
• NVIDIA GPU Driver Version (valid for GPU only) : N/A
• Issue Type( questions, new requirements, bugs) Questions

2

You can refer to the FAQ DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums to access GPU memory and sync to CPU.

4

Can you describe what you want to do with the image in detail?

5

Thanks for your reply !

I want to acquire images from a 360-degree camera using an image capture element such as nvv4l2camerasrc, crop the images around a specified viewing direction (ψ, θ) , send the cropped images to the pipeline, and draw them using nv3dsink.
The viewing direction (ψ, θ) is received by UDP communication and should be changed dynamically.

I considered using nvdewarper, but found that it is not possible to dynamically change the direction of the cutout, since it is specified by the parameter config-file.

Reference ↓

Is it possible to do what I want to do?
I hope you will reply when you have time!

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

We do not currently support dynamic configuration of these parameters.