cv2.VideoWriter[Gstreamer + Opencv] using Hardware Encoder with python code

Autonomous Machines Jetson & Embedded Systems Jetson Nano
#1

Hi all,
1- I used the below gstreamer element for using HW Decoder of jetson nano for RTSP stream:

gstream_elemets = (
                'rtspsrc location={rtps} latency=300 !'
                'rtph264depay ! h264parse !'
                'queue max-size-buffers=100, leaky=2 !'
                'omxh264dec enable-max-performance=1 enable-low-outbuffer=1 !'
                'video/x-raw(memory:NVMM),format=(string)NV12 !'
                'nvvidconv ! video/x-raw , format=(string)BGRx !'
                'videoconvert ! '
                'appsink'). 
cv2.VideoCapture(gstream_elemets, cv2.CAP_GSTREAMER)

and this part is corrctly work and use NVDEC. I capture the frames and do some processing on frames and then I want to save the frames in video file in disk using HW Encoder of jetson nano, How I can do this with gstremaer + opencv using python code?

2- If I have video file in disk and I want to open with opencv python and then using HW decoder for decoding the frames, How do I can like the above RTSP for source file?

#2

For 1.:
You would use a gstreamer pipeline in a cv2.VideoWriter, specifying cv2.CAP_GSTREAMER api:

... # assuming your gstreamer input defines caps for width, height and fps
cap = cv2.VideoCapture(gstream_elements, cv2.CAP_GSTREAMER)
if not cap.isOpened() :
    print("capture failed")
    exit()
w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)
fps = cap.get(cv2.CAP_PROP_FPS)
print('Src opened, %dx%d @ %d fps' % (w, h, fps))


gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! nvvidconv ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mov "
out= cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, float(fps), (int(w), int(h)))
if not out.isOpened():
    print("Failed to open output")
    exit()


while True:
    _, frame = cap.read()
    out.write(frame)
    cv2.waitKey(1)

For 2: use a gstreamer pipeline starting with filesrc (here displaying into X):

gst-launch-1.0 filesrc location=test.mov ! qtdemux ! video/x-h264 ! omxh264dec ! nvvidconv ! videoconvert ! xvimagesink
#3

Thanks.

This is the sudo tegrastatus output, but I don’t see NVENC, the encoder is done on CPU.

RAM 685/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [29%@1428,9%@1428,22%@1428,26%@1428] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@40C PMIC@100C GPU@38.5C AO@44C thermal@39C POM_5V_IN 2378/2037 POM_5V_GPU 0/0 POM_5V_CPU 807/557
RAM 682/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [27%@1224,21%@1224,21%@1224,15%@1224] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@40C PMIC@100C GPU@38.5C AO@44C thermal@39C POM_5V_IN 2025/2037 POM_5V_GPU 0/0 POM_5V_CPU 687/560
RAM 683/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [30%@403,12%@403,17%@403,20%@403] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36C CPU@39.5C PMIC@100C GPU@38C AO@44C thermal@39.25C POM_5V_IN 1869/2034 POM_5V_GPU 0/0 POM_5V_CPU 447/557
RAM 683/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [27%@518,11%@518,26%@518,25%@518] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@40C PMIC@100C GPU@38.5C AO@43.5C thermal@39C POM_5V_IN 1907/2031 POM_5V_GPU 0/0 POM_5V_CPU 447/555
RAM 684/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [28%@518,14%@403,21%@518,23%@518] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@39.5C PMIC@100C GPU@38.5C AO@43.5C thermal@39C POM_5V_IN 1907/2029 POM_5V_GPU 0/0 POM_5V_CPU 446/553
RAM 683/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [29%@518,32%@518,18%@518,11%@518] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@39.5C PMIC@100C GPU@38.5C AO@44C thermal@39.25C POM_5V_IN 1947/2027 POM_5V_GPU 0/0 POM_5V_CPU 486/552
RAM 684/3964MB (lfb 500x4MB) SWAP 0/10174MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [28%@518,14%@518,15%@518,26%@518] EMC_FREQ 1%@1600 GR3D_FREQ 0%@76 APE 25 PLL@36.5C CPU@40C PMIC@100C GPU@38.5C AO@44C thermal@39.25C POM_5V_IN 1947/2026 POM_5V_GPU 0/0 POM_5V_CPU 446/550

and another problem is also there, the output file size is zeros. why?

and the solution 2 for filesrc also doesn’t give me frames and hang up.

#5

Hi,
Please try

gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mov "

BGR is not supported by nvvidconv. Please use videoconvert to convert it to RGBA and then send to nvvidconv.

Improving OpenCV videowritter performance in Jetson TX2
#6

Thanks.
This config is work, and the output file is all frames frames, why? I used .mp4 for input and output format.

I also want a config gstreamer for reading cv2.VideoCapture[opencv+gstreamer] with HW NVDEC.

#7

Hi,
This might help:

Displaying to the screen with OpenCV and GStreamer