How to decode H264 stream from a usb camera in Xavier NX on openCV?

I want to decode a video from a camera, but I have no idea how to use h264 hardware decoding.
My camera provide h264 30fps stream. But I don’t know how to utilize it.
For more detail, I want use OpenCV videocapture() to capture the video stream for downstream tasks. I want to use the inner hardware acceleration, but I don’t know how to deal with it. If possible, please show me some sample code.

hi,
If your source is a v4l2 source, you can refer to


and share the information first.

I have a USB camera, which could produce MJPG(with 30 fps)/YVY2(with 5-25fps, depending on resolution) and H264 (with 30fps) format video stream. I want to use it for capturing video stream and work on OpenCV .

Firstly I tried cap=cv2.VideoCapture(1) , which only gives me 5 fps 1080p video.

Then I read some relative guidance, and they suggested me using:
gst-launch-1.0 -v v4l2src device=/dev/video1 ! image/jpeg, width=1920, height=1080 ! nvjpegdec ! "video/x-raw(memory:NVMM), format=I420" ! nvoverlaysink
It works fine on command line and gave me 30 fps 1080,

But it did not work on OpenCV and gave me error.

If I try:
cv2.VideoCapture("v4l2src device=/dev/video1 ! image/jpeg, width=1920, height=1080 ! nvjpegdec ! video/x-raw, format=I420 ! nvvidconv ! video/x-raw ! videoconvert ! appsink",cv2.CAP_GSTREAMER)
It gives me 6-7fps and a little high CPU occupation rate.

Finally, I tried
cv2.VideoCapture(“v4l2src device=/dev/video1 ! image/jpeg, width=1920, height=1080 ! jpegparse ! jpegdec ! video/x-raw, format=I420 ! nvvidconv ! video/x-raw ! appsink”,cv2.CAP_GSTREAMER)
(Note that replace nvjpegdec with jpegdec )
It works with 30fps, but with high CPU/GPU occupation. Maybe it is using CPU decoding…

Is there any other methods to decode camera stream with 30fps, full 1080p resolution and low CPU occupation? Or maybe any H264 decoding methods?

Hi,
This post may be related:


Please try with io-mode=2.