How to utilize the hardware ability to improve the video decode performance?

As the official website says, NX should support “22x 1080p30 (H.264)”. However, the cpu usage is as high as 6% when I start to read one video stream by cv2.VideoCapture(s). It seems that I don’t use the nx’s ability correctly.
I have tried to use cv2.VideoCapture(s, cv2.CAP_GSTREAMER) or cv2.VideoCapture(s, cv2.CAP_FFMPEG), but it didn’t work at all.

Here are some error codes when I try to use the cv2.CAP_GSTREAMER:

/home/nvidia/.local/lib/python3.8/site-packages/torch/functional.py:505: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3490.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]

(python3:19687): GStreamer-WARNING **: 18:31:01.652: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so': /lib/aarch64-linux-gnu/libGLdispatch.so.0: cannot allocate memory in static TLS block

(python3:19687): GStreamer-WARNING **: 18:31:01.655: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvidconv.so': /lib/aarch64-linux-gnu/libGLdispatch.so.0: cannot allocate memory in static TLS block
^C
(python3:19687): GStreamer-CRITICAL **: 18:37:32.914:
Trying to dispose element capsfilter2, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.914:
Trying to dispose element capsfilter1, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.914:
Trying to dispose element h264parse0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.914:
Trying to dispose element rtph264depay0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.914:
Trying to dispose element rtpjitterbuffer0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.916:
Trying to dispose element manager, but it is in PLAYING (locked) instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(python3:19687): GStreamer-CRITICAL **: 18:37:32.916:
Trying to dispose element rtspsrc0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

Hello and welcome to the forums. Your issue looks to be related to Jetson. Can you please confirm which model? I can then move this topic to the correct forum.

Thanks,
Tom

It is jetson xavier nx.

Thanks, I will move this over to the proper category.

Hi,
For using OpenCV, certain operations are run in CPU and performance may be capped. Please check discussion in the topic:
[Gstreamer] nvvidconv, BGR as INPUT

For getting frame data in 24-bit BGR, would need to use CPU for copying and converting the frame data. This may be bottleneck of throughput. To get optimal performance, we suggest use jetson_multimedia_api and gstreamer.