I don’t think that nvvidconv is the culprit… using :
... alpha plugin -> nvvidconv as RGBA into NVMM memory -> nvvidconv as RGBA into system memory -> compositor ...
works fine.
For your display issue with compositor, it may be related to pixel-aspect-ratio. Add verbose flag (-v) to gst-launch command so that you can get more insight about it.
You may force it to 1/1 in caps before compositor inputs (nvvidconv may also do the job such as here).
On compositor output, you may also resize to your screen resolution and add pixel-aspect-ratio=1/1 into caps.
use glvideomixer instend nvcompositor,but this will use CPU and memory,and use cpu
use nvcompositor, the nvvidcov → nvcompositor the opcaity=0 will be black… It is so sad.
"appsrc "
"! video/x-raw, format=BGR "
"! videoconvert "
"! video/x-raw,format=(string)RGBA "
“! queue2”
"! alpha method=blue "
"! glvideomixer name = compos sink_0::zorder=1 sink_0::alpha=0.85 sink_1::alpha=1 sink_1::zorder=0 "
“! nvvidconv”
“! video/x-raw(memory:NVMM), format=(string)I420, width=(int)1024, height=(int)600”
“! nvv4l2h264enc "
“! rtspclientsink location=rtsp://192.168.20.99:8554/my_pipeline”
" nvarguscamerasrc”
“! video/x-raw(memory:NVMM),format=(string)NV12, width=(int)1640, height=(int)1232, framerate=(fraction)25/1”
“! queue2”
“! nvvidconv”
“! video/x-raw,format=(string)RGBA, width=(int)1024, height=(int)600 , pixel-aspect-ratio=1/1”
"! compos. ";
because the nvvidconv->nvcompositor the black problem,i use the sink_0::alpha=0.4 ,then the brightness of the frame is very dark
so i use the videobalance contrast=0.8 brightness=0.35 ,add its brightness,The effect,Since there is a dark transparent layer in the picture, the picture looks white after adjusting the brightness.
Because i use cv::VideoWrite,write one picture to video src, and The white color in the picture is a bit like noise, do you know which gst plug-in can eliminate it ?
Hi,
A quick solution is to use nvivafilter plugin and apply alpha effect to each pixel directly. Please refer to attached CUDA code. nvsample_cudaprocess.zip (2.8 KB)
I just find nvivafilter use example from generic-no-api_r2
the pipeline like this:
nvivafilter cuda-process=true customer-lib-name=libnvsample_cudaprocess.so
In my pipeline the appsrc add one BGR,the bkg is blue.
if use video/x-raw(memory:NVMM),nvvidconv → nvcompositor ,alpha it does not work, i guess in gpu the aransparency is sacrificed for computational efficiency!
I understand your suggestion,but my alpha image is appsrc from cv::VideoWrite,while i inster nvivafilter to
nvvidconv → nvcompositor for save alpha channel,
but the pipeline report error:
N:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module appsrc0 reported: Internal data stream error.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
nvivafilter may work as alpha plugin works, but nvcompositor would not be able to process it.
Attached a nvivafilter: nviva-alpha-test.tgz (4.4 KB)
just doing thresholding from hue to alpha.
tar zxvf nviva-alpha-test.tgz
cd alpha-test
# You would edit Makefile and tune for your case (opencv install path...)
make
# in my case I used opencv-4.6.0 installed into /usr/local/opencv-4.6.0, so I first added (this may not be required for your case if you have a standard opencv intall, just for reference):
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:/usr/local/opencv-4.6.0/lib:
To be precise, transparency merging is not supported in Nvmm,but I think a small change could be made to this, which would actually improve efficiency.
When the top image opacity is zero, just retain the original value of the bottom image .
for (int i = 0; i != space_mat_.rows; ++i) {
for (int j = 0; j != space_mat_.cols; ++j) {
if (space_mat_.at<cv::Vec4b>(i, j)[3] != 0) {
cv::Vec3b scalar1 = src.at<cv::Vec3b>(i, j);
cv::Vec4b scalar2 = space_mat_.at<cv::Vec4b>(i, j);
float dbAlpha = 0.68;
for (int k = 0; k != 3; ++k) {
scalar1.val[k] = scalar1.val[k] * (1 - dbAlpha) + scalar2.val[k] * dbAlpha;
}
src.at<cv::Vec3b>(i, j) = scalar1;
// if the foreground Image 's alpha =0, show the background rgb value,else the
// background color while be black,just show foreground img with black background!!!
}
}
}
I change rtsp push_stream plan,use cv::VideoCapture → blend by code like up → cv::VideoWriter. @Honey_Patouceul
Thanks for everyone !