How to convert 3840x2160 frame to 1280x1280 in gstreamer pipelines?

[jasonrichdarmawan]

My goal is to capture 3840x2160 frame and display 1280x1280 frame. The problem is, the bottom half is cut.

        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), "
        "width=(int)3840, height=(int)2160, "
        "format=(string)NV12, framerate=(fraction)21/1 ! "
        "nvvidconv flip-method=0 ! "
        "video/x-raw, width=(int)1280, height=(int)1280, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink"

The saved frame example

[Honey_Patouceul]

Not sure as I’m unable to reproduce your case (I don’t have a Nano and just tried using AGX Orin and Xavier NX running JP5):

import cv2

cap = cv2.VideoCapture("nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)21/1 ! "
        "nvvidconv ! video/x-raw, format=(string)BGRx, width=1280, height=1280 ! "
        "videoconvert ! video/x-raw, format=(string)BGR ! "
        "queue ! appsink drop=1", cv2.CAP_GSTREAMER)

if not cap.isOpened():
   print('failed to open video capture')
   exit(-1)

while True:
   ret_val, img = cap.read();
   if not ret_val:
      break

   cv2.imshow('Test', img)	
   if cv2.waitKey(1) == ord('q'):
      break
 
cap.release()

You may try to narrow down if this is related to your camera/driver, gstreamer or opencv version with its gui-backend.

Assuming you have a local display on Jetson with X GUI, does this work ?

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=21/1' ! nvvidconv ! queue ! xvimagesink

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=21/1' ! nvvidconv ! 'video/x-raw,width=1280,height=1280,pixel-aspect-ratio=1/1' ! queue ! xvimagesink

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=21/1' ! nvvidconv ! 'video/x-raw,format=BGRx,width=1280,height=1280,pixel-aspect-ratio=1/1' ! videoconvert ! queue ! xvimagesink

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=21/1' ! nvvidconv ! 'video/x-raw,format=BGRx,width=1280,height=1280,pixel-aspect-ratio=1/1' ! videoconvert ! video/x-raw,format=BGR ! videoconvert ! queue ! xvimagesink

If these work, then your issue is in opencv build or its gui backend.

[DaneLLL]

Hi,
Please try Honey Patouceul’s suggestion. And you may also try output RGBA in nvvidconv like:

... ! nvvidconv ! 'video/x-raw,format=RGBA,width=1280,height=1280' ! ...

Besides, the nvvidconv plugin is open source. You can download the source code and check further by adding debug prints and re-bulding the plugin.

[jasonrichdarmawan]

I tried your Python code. Please try to display the frame as 256x256 → "nvvidconv ! video/x-raw, format=(string)BGRx, width=256, height=256 ! ".

Perhaps your monitor resolution is 1920x1080, that’s was also my case. I thought the display frame did not have black pixels.

Regarding running it with gst-launch-1.0, I changed the width=1080,height=1080 for the display frame to match the monitor resolution. This is because the monitor resolution over 1920x1080 is too small for me to read. My monitor resolution is 1920x1080.

I also use HDMI Video Capture to USB. This is because I only have 1 monitor. To clarify, I use OBS Studio to display NVIDIA Jetson Nano B01 GUI through HDMI Video Capture device. Not sure if this affect.

All of the gst-launch-1.0 have black pixels.

Solution:

def gstreamer_pipeline(
    # Issue: the sensor format used by Raspberry Pi 4B and NVIDIA Jetson Nano B01 are different
    # in Raspberry Pi 4B, this command
    # $ libcamera-still --width 1280 --height 1280 --mode 1280:1280
    # uses sensor format 2328x1748.
    # However, v4l2-ctl --list-formats-ext do not have such format.
    capture_width=3840,
    capture_height=2160,
    display_width=640,
    display_height=360,
    framerate=21,
    flip_method=0,
):
    return (
        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM),width=(int)%d,height=(int)%d,format=(string)NV12,framerate=(fraction)%d/1 ! "
        "nvvidconv flip-method=%d left=%d right=%d top=%d bottom=%d ! "
        "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink"
        % (
            capture_width,
            capture_height,
            framerate,
            flip_method,
            capture_width/2 - display_width/2,
            capture_width/2 + display_width/2,
            capture_height/2 - display_height/2,
            capture_height/2 + display_height/2,
            display_width,
            display_height
        )
    )

Reference: R32.3.1 tx2-4g nvvidconv problem - #21 by Honey_Patouceul