Need advice: 4K video capture & writing performance with OpenCV

need some help to improve my frames-per-second performance for my OpenCV motion-detection C++ program.

Here is my setup:

  • Jetpack 4.6
  • new build of OpenCV 4.5.4-dev (see attached for verbose build info)
  • with GStreamer 1.14.5
  • NV Power Mode 6 [20W 2 core]

I want to process an incoming RTSP 4k video stream (3840x2160) at 12 fps from IP camera using hardware-assisted h.264 decode using GStreamer, then OpenCV for motion-detection using contour detection within selected ROI. Draw the bounding box around the detected motion, and then write the video frames out to an .MP4 file using hardware-assisted h.264 (or h.265) encode using GStreamer.

Using Gstreamer Pipeline to reduce CPU load; Pipeline Elements ‘nvv4l2decoder’ use Xavier hardware for h264 frame decoding and ‘nvvidconv’ is hardware accelerated converter.

std::string pipe = “rtspsrc location=rtsp://admin:passwd@ ! rtph264depay
! nvv4l2decoder ! video/x-raw(memory:NVMM),format=NV12 ! nvvidconv ! video/x-raw,format=BGRx,width=3840,height=2180
! videoconvert ! appsink”;

cv::VideoCapture capture(pipe,cv::CAP_GSTREAMER);
const int frame_width = 3840;
const int frame_height = 2160;
const int frames_per_second = 12;
cv::Size frame_size(frame_width,frame_height);

std::string motion_writer_pipe = “appsrc ! videoconvert ! video/x-raw,format=BGRx,width=3840,height=2160,framerate=12/1 !
nvvidconv ! video/x-raw(memory:NVMM),width=(int)3840,height=(int)2160,format=NV12,framerate=(fraction)12/1 !
nvv4l2h264enc preset-level=0 iframeinterval=60 control-rate=1 bitrate=5000000 !
h264parse ! qtmux ! filesink location =” + savedMotionVideoFullPath;

cv::VideoWriter motion_writer = cv::VideoWriter(motion_writer_pipe,cv::CAP_GSTREAMER,frames_per_second,frame_size,true);

All is working, to a degree, but the fps performance is way too low, and the outputted MP4 video quality is not always acceptable. All of the motion-detection-related OpenCV C++ statements use the CPU, since there are no corresponding GPU statements for the majority of the code. So the motion-detection portion of the program is all CPU.

When all motion-detection code is removed and simply read in the RTSP video stream and then write it out using above GStreamer code, the CPU is 80+% and the resulting MP4 video often has many compression artifacts as little blurry squares in large areas of the written frame. (See attached cropped photo captures of a portion of the frame).

Is this poor performace (low fps, unacceptable encoding) due to not enough CPU speed or limits of the XavierNX itself?

What kind of hardware system do I need to successfully process 4k video with OpenCV and achieve high-quality MP4 output?

Thanks for all your help…. much appreciated.
OpenCV 4.5.rtf (8.3 KB)

Since BGR is not supported by hardware converter(VIC engine), so there is additional software conversion/data copy on CPU while running with OpenCV. We would suggest use DeepStream SDK to get optimal solution. You can install the package through SDKManager and it is at


Documents are in
NVIDIA Metropolis Documentation

For running with OpenCV, the performance bottleneck is very likely to be in CPU capability. You can execute sudo tegrastats to check system loading.

I have installed DeepStream as suggested.

dk@XavierNX:~$ deepstream-app --version
deepstream-app version 6.0.0
DeepStreamSDK 6.0.0

I am following the DeepStream QuickstartGuide as found at:

I get to this point of trying to run one of the provided demo files.

From the console, while in the samples directory as stated, I get this error.

dk@XavierNX:/opt/nvidia/deepstream/deepstream-6.0/samples$ deepstream-app -c configs/deepstream-app/source30_1080p_dec_infer-resnet_tiled_display_int8.txt

Using winsys: x11
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/…/…/models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine open error
0:00:03.423494887 12908 0x23fb5a90 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/…/…/models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed

Upon checking the file open error… the requested file does not exist… but this one does:


So I am stuck, and not sure what I should do next to try to run some of the demos.

Thanks, Dave

Please execute command at:


Or the paths are wrong and model/engine files cannot be found.

The issue is that no matter which directory I execute the command within… the chosen config file specifies the model file.

The problem is the file being requested by the config file does not exist in the downloaded DeepstreamSDK6.0 files.

How to run the samples??

Please run deepstream-app at this directory

/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app$ deepstream-app -c source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.