"nvv4l2h264enc" plugin: STOP PROBLEM

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,

We have a problem with the “nvv4l2h264enc” plugin. The problem appears when a STOP is performed (from Gstreamer’s point of view), the STOP command hangs and never leaves.

To test it we use the GstDaemon tool.
We use a script where first, we define a Pipeline that uses the plugin “nvv4l2h264enc”, and then it goes through the following sequence in a loop:

  1. START the Pipeline (gstd-client pipeline_play …)
  2. waits 10 sec (sleep 10)
  3. STOP the Pipeline (gstd-client pipeline_stop …)
  4. waits 5 seconds (sleep 5).
  5. and then returns to the first step (START).

When using this script, we see that after the STOP command the script hangs, it does not always hang in the first STOP, sometimes it hangs in the second or third STOP.

If we do the same test but using the “omxh264enc” plugin instead of using the “nvv4l2h264enc” plugin, we don’t have this STOP problem.

We have found the same problem using the plugin “nvv4l2h264enc” with the “L4T R32.3.1” and with the “L4T R32.4.3”. We were also trying to change some properties of the encoder plugin, but the same problem appears.

Therefore, the problem is related to the encoder plugin “nvv4l2h264enc”, and we need it to work correctly.

We look forward to hearing from you about this problem.
Thanks for your help.

3

Hi,
Do you observe the issue with videotestsrc like this pipeline:

videotestsrc ! nvvidconv ! nvv4l2h264enc ! fakesink

Not sure which source you use. Would like to eliminate it and see if it happens with videotestsrc.

4

Hi DaneLLL,
thanks for your fast reply.

The source we were using for the test was “v4l2src”.
So following your comments, I’ve been using now a simple pipeline:
videotestsrc ! nvvidconv ! nvv4l2h264enc ! fakesink

Using this pipeline, the problem also appears. The STOP commands hangs, it took a few more STOPs commands, but the problem also appears.

Attached you can find the gstd script I’m using: Script_gstd_videotestsrc_nvv4l2h264enc.sh
Also, attached you can find the LOG file obtained when running the previous script, as you can see the test hangs when running the STOP command.

Script_gstd_videotestsrc_nvv4l2h264enc.sh (1.3 KB)

LOG_FAIL_nvv4l2h264enc_STOP_JetsonNano.txt (19.3 KB)

We look forward to hearing from you about this problem.
Thanks again for your help.

PS. Meanwhile I was answering you, I was testing “nvv4l2h265enc”, and it has the same problem. The test hangs in one STOP command.

5

Hi,
We try to run the script and hit the error:

./Script_gstd_videotestsrc_nvv4l2h264enc.sh: line 23: gstd-client: command not found

May need to install additional package for running the command.

Could you try the case to destroy the pipeline and re-initialize it? Looks like you just stop an dre-start the pipeline. This is not the verified case. After stopping the pipeline, you would need to destroy it.

6

Hi,
yes, to run the script you need to install the gstd (GStreamer Daemon).

It’s very easy to install following the instructions detailed here:

The gstd is very useful, for example it can help you to test plugins to look for memory leaks.

On the other hand, we CAN NOT do what you suggest. Our application needs to start-stop the encoding task sometimes, so we CAN NOT destroy the global pipeline every time, because, as you can guess, in our application pipeline are running several plugins at the same time and we do not destroy the pipeline when we need to stop a specific gstreamer module from the global pipeline.

Moreover I remember you what I commented when I opened this topic, when we use the OMX encoder, we do not have any problem with the stop command. At least we need/expect the NVIDIA HW Encoders (nvv4l2h264enc and nvv4l2h265enc) manage the start-stop sequence in the same way as OMX encoders do, as any other GStreamer Plugin does.

So please, look for the solution to manage the stop command properly in your encoder plugins.
Thanks for your help.

7

It may be related to gstd or to your pipeline, or plugins failing to keep sync.
You may try this simple example launching a gstreamer pipeline going from camera source → h264enc → h264dec → videosink (needs sync=false in R32.5), where you can pause the pipeline and restart (stopping and restarting also works):

#!/usr/bin/env python

import sys, os
import gi
gi.require_version('Gst', '1.0')
gi.require_version('Gtk', '3.0')
from gi.repository import Gst, GObject, Gtk


def verbose_deep_notify_cb(object, orig, pspec, component):
    """
    A quick attempt to mimic gst-launch verbose mode.
    """
    if pspec.value_type == Gst.Caps.__gtype__:
    	caps = orig.get_current_caps()
    	if caps != None:
    	   print("%s/%s/%s: caps = \"%s\"" % (object.get_name(), orig.parent.get_name(), orig.get_name(), caps.to_string()))



class GTK_Main:
    def __init__(self):
        window = Gtk.Window(Gtk.WindowType.TOPLEVEL)
        window.set_title("CamRecorder")
        window.set_default_size(100, 100)
        window.connect("destroy", Gtk.main_quit, "WM destroy")
        vbox = Gtk.VBox()
        window.add(vbox)
        hbox = Gtk.HBox()
        vbox.pack_start(hbox, False, False, 0)
        hbox.set_border_width(10)
        hbox.pack_start(Gtk.Label(), False, False, 0)
        self.button = Gtk.Button("Start")
        self.button.connect("clicked", self.start_stop)
        hbox.pack_start(self.button, False, False, 0)
        self.button2 = Gtk.Button("Quit")
        self.button2.connect("clicked", self.exit)
        hbox.pack_start(self.button2, False, False, 0)
        hbox.add(Gtk.Label())
        window.show_all()

        # Set up the gstreamer pipeline

        # IMX219 sensor on CSI-0
        #gst_str = "nvarguscamerasrc ! video/x-raw(memory:NVMM), format=NV12, width=640, height=480, framerate=30/1 ! nvv4l2h264enc ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! xvimagesink sync=false"

        # ZED camera on /dev/video1
        gst_str = "v4l2src device=/dev/video1 ! video/x-raw,format=YUY2, width=2560, height=720, framerate=30/1 ! nvvidconv ! nvv4l2h264enc ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! xvimagesink sync=false"

        self.player = Gst.parse_launch (gst_str)
        self.player.connect('deep-notify', verbose_deep_notify_cb, self)
        bus = self.player.get_bus()
        bus.add_signal_watch()
        bus.enable_sync_message_emission()
        bus.connect("message", self.on_message)

    def start_stop(self, w, data=None):
        if self.button.get_label() == "Start":
            print("Setting pipeline to READY ...")
            self.player.set_state(Gst.State.READY)
            print("Setting pipeline to PAUSED ...")
            self.player.set_state(Gst.State.PAUSED)
            print("Setting pipeline PLAYING...")
            self.player.set_state(Gst.State.PLAYING)
            self.button.set_label("Pause")
        elif self.button.get_label() == "Resume":
            print("Setting pipeline to PLAYING ...")
            self.player.set_state(Gst.State.PLAYING)
            self.button.set_label("Pause")
        else:
            print("Setting pipeline to PAUSED ...")
            self.player.set_state(Gst.State.PAUSED)
            self.button.set_label("Resume")

    def exit(self, w):
        self.player.set_state(Gst.State.NULL)
        Gtk.main_quit()

    def on_message(self, bus, message):
        t = message.type
        if t == Gst.MessageType.EOS:
            print('Received message type EOS')
            self.player.set_state(Gst.State.NULL)
            self.button.set_label("Start")
        elif t == Gst.MessageType.ERROR:
            err, debug = message.parse_error()
            print("Error: %s" % err, debug)
            self.player.set_state(Gst.State.NULL)
            self.button.set_label("Start")
	

Gst.debug_set_active(True)
Gst.debug_set_default_threshold(0)
GObject.threads_init()
Gst.init(None)
GTK_Main()
Gtk.main()
8

Dear Honey_Patoucel,
the gstd does not have any problem and you know… it’s not the problem… it works well with any GStreamer plugin we use…
The pipeline we are using is the simplest of the world, so it’s not the problem:
videotestsrc ! nvvidcon ! nvv4l2h264enc ! fakesink
fakesink uses sync=false as default as you know.
Let me insist, OMX encoders work well whe using in the same START - STOP test.
Your example isn’t useful because it’s a “pause” example, it’s not a START - STOP test.

I would appreciate NVIDIA staff take a look to this STOP problem and solve it as soon as possible.
Thanks again.

9

As said above, it also works with start/stop:

#!/usr/bin/env python

import sys, os
import gi
gi.require_version('Gst', '1.0')
gi.require_version('Gtk', '3.0')
from gi.repository import Gst, GObject, Gtk


def verbose_deep_notify_cb(object, orig, pspec, component):
    """
    A quick attempt to mimic gst-launch verbose mode.
    """
    if pspec.value_type == Gst.Caps.__gtype__:
    	caps = orig.get_current_caps()
	if caps != None:
	   print("%s/%s/%s: caps = \"%s\"" % (object.get_name(), orig.parent.get_name(), orig.get_name(), caps.to_string()))



class GTK_Main:
    def __init__(self):
        window = Gtk.Window(Gtk.WindowType.TOPLEVEL)
        window.set_title("CamRecorder")
        window.set_default_size(100, 100)
        window.connect("destroy", Gtk.main_quit, "WM destroy")
        vbox = Gtk.VBox()
        window.add(vbox)
        hbox = Gtk.HBox()
        vbox.pack_start(hbox, False, False, 0)
        hbox.set_border_width(10)
        hbox.pack_start(Gtk.Label(), False, False, 0)
        self.button = Gtk.Button("Start")
        self.button.connect("clicked", self.start_stop)
        hbox.pack_start(self.button, False, False, 0)
        self.button2 = Gtk.Button("Quit")
        self.button2.connect("clicked", self.exit)
        hbox.pack_start(self.button2, False, False, 0)
        hbox.add(Gtk.Label())
        window.show_all()

        # Set up the gstreamer pipeline
        #gst_str = "nvarguscamerasrc ! video/x-raw(memory:NVMM),format=(string)NV12,width=(int)640,height=(int)480, framerate=30/1 ! nvv4l2h264enc ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! xvimagesink sync=false"
        gst_str = "v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=2560,height=720, framerate=30/1 ! nvvidconv ! nvv4l2h264enc ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! xvimagesink sync=false"
        self.player = Gst.parse_launch (gst_str)
        self.player.connect('deep-notify', verbose_deep_notify_cb, self)
        bus = self.player.get_bus()
        bus.add_signal_watch()
        bus.enable_sync_message_emission()
        bus.connect("message", self.on_message)

    def start_stop(self, w, data=None):
        if self.button.get_label() == "Start":
            print("Setting pipeline to READY ...")
            self.player.set_state(Gst.State.READY)
            print("Setting pipeline to PAUSED ...")
            self.player.set_state(Gst.State.PAUSED)
            print("Setting pipeline PLAYING...")
            self.player.set_state(Gst.State.PLAYING)
            self.button.set_label("Stop")
        else:
            print("Setting pipeline to NULL ...")
            self.player.set_state(Gst.State.NULL)
            self.button.set_label("Start")

    def exit(self, w):
        self.player.set_state(Gst.State.NULL)
        Gtk.main_quit()

    def on_message(self, bus, message):
        t = message.type
        if t == Gst.MessageType.EOS:
            print('Received message type EOS')
            self.player.set_state(Gst.State.NULL)
            self.button.set_label("Start")
        elif t == Gst.MessageType.ERROR:
            err, debug = message.parse_error()
            print "Error: %s" % err, debug
            self.player.set_state(Gst.State.NULL)
            self.button.set_label("Start")
	

Gst.debug_set_active(True)
Gst.debug_set_default_threshold(0)
GObject.threads_init()
Gst.init(None)
GTK_Main()
Gtk.main()

[EDIT: Further checked with gstd, I see no problem in my case (using Xavier-NX):

gstd
gst-client pipeline_create test videotestsrc ! nvvidconv ! nvv4l2h264enc ! nvv4l2decoder ! nvvidconv ! xvimagesink  sync=false

gst-client pipeline_play test
...
gst-client pipeline_stop test

gst-client pipeline_play test
...
gst-client pipeline_stop test

It works and reports success.]

10

Hi again,
sorry but does not work… It is not failing in the first or second STOP… In this case it’s failing after 20 or 25 STOPs…
Based in your example, I’m testing the following example (see attached file):

start_stop_test.py (3.5 KB)

11

Hi @GasparGS,
Could you try the pipeline in play and null state switch?

videotestsrc ! nvvidconv ! nvv4l2h264enc ! fakesink

We would suggest go to null state instead of only stop state. The pipeline in start_stop_test.py is complicated. May break down the pipeline for a try.

12

Hi DaneLLL,
I’ve updated the L4T version of my Jetson Nano to R32.5.0, and I can confirm that the STOP problem of the HW encoders that appeared in previous L4T versions, does not appear in this version.
It does not appear with the gstd test, nor with the python file test.
I’ve been doing start-stop tests with both encoders, nvv4l2h264enc and nvv4l2h265enc, so this L4T version solves the STOP problem.

1 Like