Recordscreen and encode H264 video

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Our program need recordscreen, encode to H264 stream and push to a external server.

Is there any way we can capture screen to like RGB or YUV image quickly, then we can encode them to H264 video?

2

You may try to do that with gstreamer (this would record X screen @30fps, rescale with HW into 720p, encode with HW into H264, put into a mp4/mov container and save into file. Stop with Ctrl-C):

gst-launch-1.0 ximagesrc use_damage=0 ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,framerate=30/1' ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_h264.mp4 -e
3

Thanks, @Honey_Patouceul

After we capture the screen and get an image, we will need draw something on the image, then encode it to a video. So we may not use the above gstreamer command. Dont know if we can have some API to capture screen.

4

You may try using XLib. Have a look to how ximagesrc uses it;

Also note that you can use a gstreamer pipeline for capturing from screen, process frame, then use another gstreamer pipeline for encoding and saving/streaming. It would be very easy with OpenCV.

5

Great. We do need push the encoded H264 video to remote by udp and also need save the H264 video to local with mp4 file.

6

You would try:

#!/usr/bin/env python

import cv2
print(cv2.__version__)
#print(cv2.getBuildInformation())

# This pipeline captures screen @ 30 fps, resize to 1280x720, and convert into BGR 
cap = cv2.VideoCapture("ximagesrc use_damage=0 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)NV12, width=1280,height=720,framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! queue ! appsink drop=1", cv2.CAP_GSTREAMER)
if not cap.isOpened():
   print('Failed to open camera')
   exit(-1)

# This pipeline converts to NV12 format into NVMM memory, encodes into h264, 
# then forks into one branch streaming as RTP-MP2T to localhost port 5004, 
# and one branch putting into MP4/MOV container and save to file.
out = cv2.VideoWriter("appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=15 ! tee name=t ! queue ! h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink host=127.0.0.1 port=5004      t. ! queue ! h264parse ! qtmux ! filesink location=test.mp4 ", cv2.CAP_GSTREAMER, 0, 30.0, (1280,720)) 
if not out.isOpened():
   print('Failed to open output writer')
   cap.release()
   exit(-2)


# Loop for 100 seconds
for i in range(3000):
	# Read image from capture
	ret_val, img = cap.read();
	if not ret_val:
		break

	# Your processing....here drawing a green rectangle
	cv2.rectangle(img, (100,100), (400,400), (0,255,0), thickness=2, lineType=2)

	# Push processed image into writer
	out.write(img);
	
out.release()
cap.release()
7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.