Capture high-res photo with preview in Python

Hi there,

I am trying to build a camara program in Python that shows a preview window and captures a photo when pressed a key. The preview is supposed to be 720p for performance consideration. But the captured photo should be 4056*3040 in size. I am using Jetson Nano and a CSI IMX477 camera.

This is like simulating what nvgstcapture command does for capturing a photo. I tried to read the source code of nvgstcapture, but end in vain since I am no expert in gstream programming.

I have tried a few approaches with gstream.

  1. Use gst-launch-1.0 command to read in 4056*3040 camera source, then use tee to duplicate the stream. One keep the original size, one use nvvidconv to convert to 720p. Both stream are encoded with h264 and sent to a UDP port. Then use python code to listen to UDP, decode the 720p stream, and show preview. Capture a frame of HQ stream when pressed key.
gst-launch-1.0 nvarguscamerasrc ! \
'video/x-raw(memory:NVMM), format=NV12, width=4056, height=3040' ! \
tee name=t ! \
queue ! nvv4l2h264enc insert-sps-pps=true ! h264parse !  rtph264pay pt=96 ! \
udpsink host=127.0.0.1 port=5004 sync=false t. ! \
queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=1080, height=720' ! \
nvv4l2h264enc insert-sps-pps=true ! h264parse !  rtph264pay pt=96 ! \
udpsink host=127.0.0.1 port=5005 sync=false
  1. Use Python to read in full-size camera stream, resize each frame to 720p for preview, save full-size photo when pressed key.
camera = cv2.VideoCapture(
        "nvarguscamerasrc! "
        "video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, framerate=(fraction)30/1 ! "
        "nvvidconv flip-method=2 ! "
        "video/x-raw, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink",
        cv2.CAP_GSTREAMER
    )

# resize each frame and show preview

Both approaches suffers from a high preview latency of ~2s.

I know there’s already a post asking a similar question in C/C++:
https://forums.developer.nvidia.com/t/how-to-turn-off-auto-capture-when-gstreamer-pipeline-runs-only-on-demand/66539
Yet the final answer seems to be compiling a copy of modified source code of nvgstcapture.
I want to know if this is possible using Python?

Regards

Hi,
Please try to set small idrinterval for a try. Check if there is improvement in latency. Can refer to commands in
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

For JPEG encoding in OpenCV, we can call cv2.imwrite() but it has to copy data from NVMM buffer to CPU buffer. The resolution is 4056*3040 so this execution may impact performance. This is not recommended.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.