Hi,
I’m trying to customise gstreamer plugin so that I can do some image processing.
for example, I want to remap or warp the input buffer to the output buffer.
and in the end, composit multiple remaped camera feed to the one parnoramic frame.
in this case, should I write a custom deepstream plugin based on gstdsexample changing transform_ip or transform function?
or should I write use the nvivafilter with custom cuda kernel (.cu)?
Basically I want to do the real-time video pipeline for below.
multi nvarguscamera --> panorama (this will be the custom plugin) --> encoding the panorama (gstreamer is enough)
--> detection from panorma(deepstream needed)
So in this case, I should use deepstream for the best performance?
I haven’t check the nvdsvideotemplate before, but I was referencing only dsexample,
In short, I should use nvdsvideotemplate for the customisation.
I checked the plugin and I’m not sure if this is what I need or not.
let me elaborate more about the pipeline
Left nvarguscamera(NV12) -->
)--> Custom plugin (remap each input frames and blend into one frame) --> downstream Encoder --> ...
Right nvarguscamera(NV12) -->
My pipeline, as you can see from the image above, remaps(or warps) the left and right streams.
And then composites them into one stitched image which will overlap and blend some part of remaped frames.
in nvdspreparocess sample code, I don’t know where I can overlap the input frames and then output it into one frame.
and when can I implement the remapping (as a cuda kernel)? I see the prepare_tensor in nvdspreprocess_imple.cpp. but I’m not sure because the point of the customisation is image processing, not the detection…
All DeepStream plugins work on batch. You need to use nvstreammux to batch the two sources from the two cameras. The nvdspreprocess sample library supports tensors with ROIs, if you configure the whole picture as the ROI, then the tensor is the whole image.
You don’t need to implement your mapping unless your model’s input tensor needs special preprocessing. Current nvdspreprocess sample libary support scaling, normalization and format conversion preprocessing.
Please read the sample code, every detail is available in the source codes.
Fiona,
My point of doing this is to implement the custom image processing(warping of two input → blending to one panorama). please refer the image that I attached above.
