Gstreamer seeking deadlock

We have something else thats not working and may be strongly be linked to this problem :

   auto const [framerate_num, framerate_den] = [&]() -> std::pair<int64_t, int64_t> {
        if (auto const caps = GetElement(element_name).GetCaps(); caps && GST_IS_CAPS(caps.get())) {
            if (GstStructure const* structure = gst_caps_get_structure(caps.get(), 0); structure) {
                if (GValue const* framerate = gst_structure_get_value(structure, "framerate"); framerate) {
                    if (G_VALUE_TYPE(framerate) == GST_TYPE_FRACTION_RANGE) {
                        framerate = gst_value_get_fraction_range_min(framerate);
                    }

                    if (framerate) {
                        return {gst_value_get_fraction_numerator(framerate), gst_value_get_fraction_denominator(framerate)};
                    }
                }
            }
        }

        return {};
    }();

Always return {0,1}

Can you confirm the JetPack version is r35.2.1?

What is the element? How can we know what is happening with just a piece of code? Can you provide a simple instruction of duplicating this issue in our Jetson device?

Yes

head -1 /etc/nv_tegra_release

R35 (release), REVISION: 2.1, GCID: 32413640, BOARD: t186ref, EABI: aarch64, DATE: Tue Jan 24 23:38:33 UTC 2023


You can find here a sample of code that is reproducing the issue.

The Deadlock doesn’t happens always but most of the time.

I have linked as well a test video that was triggering it.

CMakeLists.txt (1.3 KB)
main.cpp (13.1 KB)
main.hpp (2.6 KB)

Since there is no appsink sample consumer implementation in your code, can you use the following pipeline instead?
gst-launch-1.0 filesrc location=xxx.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=UYVY ! appsink drop=TRUE max-buffers=2 enable-last-sample=FALSE wait-on-eos=FALSE

Thank you for your answer,

by applying what you are suggesting

--- a/main.cpp
+++ b/main.cpp
@@ -300,6 +300,10 @@ int main(int argc, char * argv[]) {
   r_source.SetProperty("location", argv[1]);

   Element *p_output_ = &pipeline_.Add("appsink", "video-output");
+  p_output_->SetProperty("drop", true);
+  p_output_->SetProperty("max-buffers", 2);
+  p_output_->SetProperty("enable-last-sample", false);
+  p_output_->SetProperty("wait-on-eos", false);

   // clang-format off
     p_output_->SetProperty("caps", gst_caps_new_simple(

I still get the deadlock :

Opening in BLOCKING MODE
Dynamic pad created, linking demuxer/decoder
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261

(gstreamer_deadlock:95924): GLib-GObject-CRITICAL **: 16:41:44.979: g_object_unref: assertion ‘G_IS_OBJECT (object)’ failed
INVALID framerate_num HERE : 0, 1
DEADLOCK HERE

FYI : I did not share our consumer implementation because this part of code is never reached (due to the deadlock) that doesn’t make it pertinent for your investigation.

Thank you

Hi,
Please check this sample and give it a try:
Deadlock when try to seek in pipeline

Hello,

What do you mean by “this sample”, that simple :

filesrc location=video.mkv ! matroskademux name=demuxer demuxer.video_0 ! identity name=tmp ! nvv4l2decoder ! nvvidconv ! video/x-raw ! appsink name=sink emit-signals=true sync=false

thank you.

Hi,
The user shares a sample seek_bug_min_ex.zip and we can run it without hitting deadlock. Would suggest you give it a try.

Unfortunatly, I cant generate the video.mkv from this sample.

As mentioned by you on this thread: Video encoder nvv4l2vp9enc in Orin - #3 by DaneLLL, vp9encodding is not supported.

Could you please give me a sample so I can try this ?

Hi,
You can modify the samplle to test h264 or h265.
In generate_video.sh, please change the pipeline to

gst-launch-1.0 videotestsrc num-buffers=4500 ! video/x-raw, format=BGR, width=1920, height=1080, framerate=15/1 ! videoconvert ! nvvidconv ! "video/x-raw(memory:NVMM), format=NV12" ! nvv4l2h264 maxperf-enable=1 control-rate=0 bitrate=8000000 ! h264parse ! matroskamux ! filesink location=video.mkv

And run the pipeline in test sample:

    std::string seek_pipeline =
        "filesrc location=" + video_file + " ! "
        "matroskademux name=demuxer demuxer.video_0 ! "
        "h264parse ! nvv4l2decoder ! "
        "nvvidconv ! "
        "video/x-raw ! "
        "appsink name=sink emit-signals=true sync=false";

With your simple I couldn’t reproduce the deadlock,

However, by applying the following patch on main.cpp :

diff -u main.cpp ../seek_bug_min_ex_/main.cpp
--- main.cpp    2021-07-28 17:09:43.000000000 +0200
+++ ../seek_bug_min_ex_/main.cpp        2023-03-27 17:43:05.874309635 +0200
@@ -7,6 +7,7 @@
 #include <mutex>
 #include <thread>

+
 std::atomic_bool done{false};

 void done_handler(int) { done = true; }
@@ -65,10 +66,10 @@
     // clang-format off
     std::string seek_pipeline =
         "filesrc location=" + video_file + " ! "
-        "matroskademux name=demuxer demuxer.video_0 ! "
-        "nvv4l2decoder ! "
+        "qtdemux ! "
+        "h264parse ! nvv4l2decoder ! "
         "nvvidconv ! "
-        "video/x-raw ! "
+        "video/x-raw,format=UYVY ! "
         "appsink name=sink emit-signals=true sync=false";
     // clang-format on

@@ -111,6 +112,22 @@
       }
       gst_message_unref(msg);

+      auto sink = gst_bin_get_by_name(GST_BIN(pipeline), "sink");
+      auto pad = gst_element_get_static_pad (sink, "sink");
+      auto caps = gst_pad_get_current_caps(pad);
+      if (caps && GST_IS_CAPS(caps)) {
+         GstStructure const *structure = gst_caps_get_structure(caps, 0);
+
+         GValue const *framerate = gst_structure_get_value(structure, "framerate");
+         if (G_VALUE_TYPE(framerate) == GST_TYPE_FRACTION_RANGE) {
+            framerate = gst_value_get_fraction_range_min(framerate);
+         }
+         std::cout << "framerate : " << gst_value_get_fraction_numerator(framerate) << " " << gst_value_get_fraction_denominator(framerate) << std::endl;
+      }
+
+      gst_object_unref(pad);
+      gst_caps_unref (caps);
+
       std::cout << "start seek" << std::endl;
       // FIXME freezing here
       bool seek_result = gst_element_seek_simple(

For floating point framerate (15.02), I stil get a wrong framerate number : 0/1.

EDIT : By adding a sleep of 100ms just before the seek, We also get the deadlock with your sample (with both : an integral framerate of 15fps and a video of15.02fps).

diff -u seek_bug_min_ex/main.cpp seek_bug_min_ex_/main.cpp
--- seek_bug_min_ex/main.cpp    2023-03-27 18:18:32.471061893 +0200
+++ seek_bug_min_ex_/main.cpp   2023-03-27 18:26:09.570088425 +0200
@@ -111,7 +112,7 @@
       }
       gst_message_unref(msg);

-
+      std::this_thread::sleep_for(std::chrono::milliseconds{100});
       std::cout << "start seek" << std::endl;
       // FIXME freezing here
       bool seek_result = gst_element_seek_simple(

Hi,
Please check if you can share the video file. It seems like it is specific to the video file you are testing.

Hi,

The file is bigger than 100Mb so I can’t upload it,

But I can reproduce the deadlock with the sleep above and the file generated by your pipeline :

gst-launch-1.0 videotestsrc num-buffers=4500 ! video/x-raw, format=BGR, width=1920, height=1080, framerate=15/1 ! videoconvert ! nvvidconv ! "video/x-raw(memory:NVMM), format=NV12" ! nvv4l2h264enc maxperf-enable=1 control-rate=0 bitrate=8000000 ! h264parse ! matroskamux ! filesink location=video.mkv

Hi,
Please try to do seeking in GST_STATE_PLAYING state. We don’t see any issue in adding the 100ms sleep if the pipeline is in PLAYING state. Please give it a try.

Hi,

Unfortunatly, we need to stay on GST_STATE_PAUSED state.

In fact, we need to process video frame by frame and we want to unsure minimal memory usage.

Hi,
The use-case looks advancing and it may be better to use jetson_multimedia_api. You may refer to

/usr/src/jetson_multimedia_api/samples/00_video_decode

You can get decode YUV data frame by frame in dec_capture_loop_fcn(). It is more flexible for accessing/processing each decoded buffer.

gstreamer is a solution for simple video playback or using DeepStream SDK. If your use-case is advancing, we would suggest use jetson_multimedia_api.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.