Gstreamer encoding errors opencv - jetpack 4.4.1

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
, ,
1

Getting these errors inconsistently when writing frames to mp4 using opencv and gstreamer on NX and 4.4.1 - It used to work just fine on older version of jetpack.

The pipline is as follows

pipeline = "appsrc ! videoconvert ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)NV12, framerate="
            + std::to_string(static_cast<int>(video_data.fps)) + "/1, width=" + std::to_string(frame_width_) +
            ", height="+ std::to_string(frame_height_) + " ! " + "omxh264enc bitrate=8000000 ! h264parse !" +
            " mp4mux ! filesink location=" + video_file.path  + " sync=false";
cv::VideoWriter writer(pipeline, fourcc, static_cast<int>(video_data.fps), cv::Size(frame_width_, frame_height_));
.
.
.            
writer.write(img);

Sometimes it works but mostly I get this result for every frame pushed into the opencv writer. The images being input are valid as I can visualize them separately.

WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module appsrc2 reported: Internal data stream error.
[ WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:1] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

Any idea what is going on? The fact that it isn’t every time is confusing to me. I am also running hw jpeg encoding and decoding on other threads and that is working.

When it works I just see this:

Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 66, Level = 40
2

You may have a look to this example. You would just change shmsink to mp4mux + filesink.
Be sure you push frames at said rate. If get framerate from source, you would have to push every frame to writer.
Otherwise you would have to adjust writer fps.

1 Like
3

Thanks for the suggestion. Adding ! video/x-h264, stream-format=byte-stream ! to my pipeline seems to have solved it :)

FYI I don’t need to push frames at said rate. This is being written to a file so the encoder can handle frames being delivered faster and written to file with the correct framerate.

Thanks