We are several issues using GStreamer running continuously on an NVIDIA Jetson Xavier. We run GStreamer through python using OpenCV and have been experiencing several different issues causing GStreamer to crash and disrupting our process. It would be great if we could get some help resolving these issues so we can reliably run GStreamer with our application without it crashing.
Workflow
Python application utilizing OpenCV and other libraries that will:
Read frames from the camera
Encode frames to h264 using OpenCV + GStreamer with omxh264enc to 1 min video chunks and upload to cloud
Perform inference using a machine learning model and other processes
System information
Jetson AGX Xavier
R32 (release), REVISION: 1.0, GCID: 14531094, BOARD: t186ref, EABI: aarch64, DATE: Wed Mar 13 07:41:08 UTC 2019
gst-launch-1.0 version 1.14.5
GStreamer 1.14.5
Python version 3.6.8
OpenCV version 3.3.1
GStreamer pipeline
Here is the gstreamer pipeline we are using for cv2.VideoWriter:
appsrc ! videoconvert ! omxh264enc profile=2 preset-level=1 control-rate=2 bitrate=10000000 ! video/x-h264, streamformat=(string)byte-stream ! h264parse ! qtmux ! filesink location=test.mp4
Python code:
fps = 16.0
frame_width = 2448
frame_height = 2048
frame_writer = cv2.VideoWriter(f“appsrc ! videoconvert ! omxh264enc profile=2 preset-level=1 control-rate=2 bitrate=10000000 ! video/x-h264, streamformat=(string)byte-stream ! h264parse ! qtmux ! filesink location=test.mp4”, 0, fps, (frame_width, frame_height))
frame_writer.write(frame)
frame_writer.release()
Error
And this error happened randomly:
Framerate set to : 16 at NvxVideoEncoderSetParameterH264: Profile = 77, Level = 40 NvMMLiteVideoEncDoWork: Surface resolution (1 x 7864320) smaller than encode resolution (2448 x 2080)
VENC: NvMMLiteVideoEncDoWork: 4207: BlockSide error 0x4
Event_BlockError from 330BlockAvcEnc : Error code - 4
This error will block the calling thread indefinitely. I cannot reproduce it since it happens randomly.
GStreamer pipeline
After looking at this post https://devtalk.nvidia.com/default/topic/1056389/jetson-agx-xavier/error-encoding-with-gstreamer-and-omxh264enc/, we also tried to use nvv4l2h264enc with the following pipeline:
appsrc ! videoconvert ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)I420 ! nvv4l2h264enc profile=2 preset-level=1 control-rate=1 bitrate=10000000 ! video/x-h264, stream-format=(string)byte-stream ! h264parse ! qtmux ! filesink location=test.mp4
Error
And this error happened randomly:
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 68, SyncPointValue = 0)
H264: Profile = 77, Level = 0 NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 70, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 68, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22 NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 71, SyncPointValue = 0) NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 68, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 70, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 68, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 71, SyncPointValue = 0)
This error also blocks the calling thread indefinitely.
And it keeps printing following CRITICAL error even if the video is encoded correctly
(python3:12628): GStreamer-CRITICAL **: 21:16:56.004: gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed
We also tried to run it at command line
gst-launch-1.0 videotestsrc ! 'video/x-raw,width=(int)640,height=(int)480,format=(string)I420' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvv4l2h264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e
Here is the output
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is PREROLLING ...
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:06.822098131
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
(gst-launch-1.0:31242): GStreamer-CRITICAL **: 14:54:20.872: gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed
Setting pipeline to NULL ...
Freeing pipeline ...