I am now doing a project that requires 4-way 1920x1080 30fps decoding and the decoded video is stitched together for display.
I now use opencv to complete this function, but I found that when I use filesrc for 4-way decoding and splicing, cpu occupancy rate is full, only about 7 frames per second.
What I need is video stitching after at least 30 frames of smooth display. Will the TX1 platform in terms of performance can meet my needs, I know VIC module can do this thing, but I did not find any call on the VIC module API, Multi Media API has some VIC control code, but no video stitching related API
Please help me, or give me some advice, thanks a lot !
I use the opencv version is 3.2.0, the following is the compiler configuration:
cmake -DWITH_GSTREAMER=ON -DWITH_OPENGL=ON -DWITH_GTK_2_X=ON -DWITH_QT=ON -DWITH_CUDA=ON -DWITH_CUBLAS=ON -DWITH_NVCUVID=ON -DCUDA_ARCH_BIN="5.3" -DCUDA_ARCH_PTX="" -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCUDA_FAST_MATH=ON -DCMAKE_INSTALL_PREFIX=/home/ubuntu//workplace/test/opencv-3.2.0 ..
Here is my opencv test program:
#include <stdio.h>
#include <unistd.h>
#include <stdint.h>
#include <opencv2/opencv.hpp>
using namespace cv;
using namespace std;
int main(int argc, char** argv )
{
cv::VideoCapture cap("filesrc location=camera_1080P30_8M.mp4 ! decodebin ! videoconvert ! video/x-raw, format=(string)BGR, framerate=30/1 ! appsink -e");
if(!cap.isOpened()){
std::cout<<"read cam failed"<<std::endl;
return -1;
}
cv::VideoCapture cap1("filesrc location=camera_1080P30.mp4 ! decodebin ! videoconvert ! video/x-raw, format=(string)BGR, framerate=30/1 ! appsink -e");
if(!cap.isOpened()){
std::cout<<"read cam failed"<<std::endl;
return -1;
}
cv::VideoCapture cap2("filesrc location=test.mp4 ! decodebin ! videoconvert ! video/x-raw, format=(string)BGR, framerate=30/1 ! appsink -e");
if(!cap.isOpened()){
std::cout<<"read cam failed"<<std::endl;
return -1;
}
cv::VideoCapture cap3("filesrc location=test1.mp4 ! decodebin ! videoconvert ! video/x-raw, format=(string)BGR, framerate=30/1 ! appsink -e");
if(!cap.isOpened()){
std::cout<<"read cam failed"<<std::endl;
return -1;
}
Mat frame,frame1,frame2,frame3;
Mat combine1,combine2,combine;
namedWindow("edges",WINDOW_OPENGL);
int cnt = 0;
for(;;)
{
const int64 start = getTickCount();
cap >> frame; // get a new frame from file
cap1 >> frame1; // get a new frame from file
cap2 >> frame2; // get a new frame from file
cap3 >> frame3; // get a new frame from file
hconcat(frame,frame1,combine1); //Video for horizontal stitching
hconcat(frame2,frame3,combine2);//Video for horizontal stitching
vconcat(combine1,combine2,combine);//Video for vertical stitching
imshow("edges", combine);
waitKey(1);
const double timeSec = (getTickCount() - start) / getTickFrequency();
cout << "Time : " << timeSec * 1000 << " ms" << endl;
}
return 0;
}
Run-time cpu usage of all 100%:
RAM 2408/3995MB (lfb 61x4MB) cpu [100%,100%,100%,100%]@1734 GR3D 27%@76 EDP limit 0
RAM 2416/3995MB (lfb 61x4MB) cpu [100%,100%,100%,100%]@1734 GR3D 0%@153 EDP limit 0
RAM 2407/3995MB (lfb 61x4MB) cpu [100%,99%,100%,100%]@1734 GR3D 18%@76 EDP limit 0
RAM 2407/3995MB (lfb 61x4MB) cpu [100%,100%,100%,100%]@1734 GR3D 6%@76 EDP limit 0
RAM 2408/3995MB (lfb 61x4MB) cpu [100%,100%,100%,100%]@1734 GR3D 0%@460 EDP limit 0
RAM 2408/3995MB (lfb 61x4MB) cpu [100%,100%,100%,100%]@1734 GR3D 14%@76 EDP limit 0
The following figure shows the current display:
External Media