Camera fps too low on Xavier

hi,
I’m using a usb camera on Jetson Xavier with opencv,but the fps is too low, it’s only 5, my code is like this

cv::VideoCapture cap;
cv::Mat img;
cap.open(0);
while(true)
{
cap >> img;
cv::imshow(“ret”, img);
}

and I get warning “cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1”

Hi,

Please run with sudo jetson_clocks to enable high cpu clk and then run your app again.

You may also try adding :

cv::waitKey(1);

after cv::imshow, so that opencv drawing thread can be scheduled.

hi,
thanks for your reply,
It seems that jetson_clocks doesn’t work,I still get 5 fps when I run sudo jetson_clocks,
when I set CAP_PROP_FRAME_WIDTH and CAP_PROP_FRAME_HEIGHT as 640x480, it looks like faster
but when I print CAP_PROP_FPS it’s still 5

I have cv::waitKey(200) in loop, fps is still 5

If you wait for 200ms for each frame, you cannot get more than 5 fps. I’d suggest to use 1ms for now.

You may check the available modes for your sensor with:

v4l2-ctl -d0 --list-formats-ext

and post these for better advice. (v4l2-ctl is provided by apt package v4l-utils)
You may be able to set width, height and fps according to the mode you want after cap.open().
Also note that you may have to perform a read for fps value read from CAP_PROP_FPS to be updated.

Also note that imshow is not very fast on jetson for high resolutions. You may try a videoWriter with a gstreamer pipeline to nvoverlaysink instead:

cv::VideoWriter gst_testsink;
gst_testsink.open("appsrc ! queue ! videoconvert ! video/x-raw, format=RGBA ! nvvidconv ! nvoverlaysink ", cv::CAP_GSTREAMER, 0, fps, cv::Size(width, height));

in the loop, you would remove imshow and waitKey and use instead:

while(1)
{
   if (!cap.read(img)) {
       std::cout<<"Capture read error"<<std::endl;
       break;
   }
   gst_testsink.write(img);
}
gst_testsink.release();
cap.release();

hi, thanks for your reply, here is what I got from v4l2-ctl

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 432x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x184
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 176x144
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 160x120
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 960x720
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.032s (31.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.033s (30.000 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 1920x1080
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 432x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x184
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 176x144
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 160x120
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.048s (21.000 fps)
		Size: Discrete 960x720
			Interval: Discrete 0.091s (11.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.077s (13.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.111s (9.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.167s (6.000 fps)


I also modified my code like this

int main()
{
	VideoCapture cap(0);
	cv::VideoWriter gst_testsink;
	gst_testsink.open("appsrc ! queue ! videoconvert ! video/x-raw, format=RGBA ! nvvidconv ! nvoverlaysink ", cv::CAP_GSTREAMER, 0, 3, cv::Size(640, 480));
	cv::Mat img;

	double fps=0;

	cap.set(cv::CAP_PROP_FRAME_WIDTH,640);
	cap.set(cv::CAP_PROP_FRAME_HEIGHT,480);
	cap.set(cv::CAP_PROP_FPS,30);

	while(true)
	{
	
		if (!cap.read(img)) {
       		std::cout<<"Capture read error"<<std::endl;
       		break;
   		}
   		gst_testsink.write(img);
		//imshow("CamShow",frame);
		
		fps = cap.get(cv::CAP_PROP_FPS);
		cout<<"FPS:"<<fps<<endl;
		

		waitKey(1);
	}

	gst_testsink.release();
	cap.release();
	
	return 0;
}

when I ran the code, the output is

xavier2@xavier2-desktop:~/code/videoCapture/build$ ./videoCapture 
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5
FPS:5

when I set width and height as 640x480 and used gstreamer pipeline I still got 5 fps,
but when I used imshow instead, it became much faster, the print of CAP_PROP_FPS was always 5,
It seemed that cap.set(cv::CAP_PROP_FPS,30) didn’t work

You may try a gstreamer pipeline for capture as well.
First check this works:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! xvimagesink

If ok for opencv you would make a pipeline into BGR format for appsink

VideoCapture cap("v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! video/x-raw, format=BGR ! appsink", cv::CAP_GSTREAMER);

For higher resolutions, videoconvert may be slow, so you could also try to perform YUV->BGR conversion with HW:

VideoCapture cap("v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink", cv::CAP_GSTREAMER);