X264enc causes pipeline to hang in DS 6.2, DS 6.3

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU): Jetson Orin Nano
**• DeepStream Version: 6.2/6.3
**• JetPack Version (valid for Jetson only): 5.1.2(L4T 35.4.1)
**• Issue Type( questions, new requirements, bugs): bug

My pipeline is as follows:
uridecodebin → nvstreammux → nvinfer → nvtracker → nvdsanalytics → nnvvideoconvert → capsfilter → queue →
nvosd → nvvideoconvert → capsfilter → x264enc → h264parse → splitmuxsink

The pipeline works in deepstream version 6.1.1, however, in deepstream 6.2 or 6.3, it hangs for about 2 minutes before starting to work.

Referring to x264enc, I have attempted to set the tune property of x264enc to zero-latency, or increase the buffer size of the queue elements in my pipeline, unfortunately, none of these settings worked.

I think this is a bug in ds 6.2 and ds 6.3. Please help me check it.

Let’s narrow down the scope first. Could you run that with the pipeline below first?

uridecodebin → nvstreammux → nvinfer->nvvideoconv->nvdsosd->nvvidconv1->capfilt->x264enc->h264parse->splitmuxink

You can also refer to the 270638 topic.

Thank you for your reply,

I tested your suggested pipeline and the result is the same. I also checked the topic you refer to, played around with different settings for x264enc element but as mentioned, the pipeline still hangs.

Could you attach your command pipeline or the code to us?

Hi @yuweiw,
Docker image: nvcr.io/nvidia/deepstream:6.3-samples

gst-launch-1.0 uridecodebin uri="file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4" nvstreammux width=1920 height=1080 batch-size=1 \
! nvinfer config-file-path=dstest1_pgie_config.txt ! nvvideoconvert  \
! capsfilter caps="video/x-raw(memory:NVMM), format=RGBA" ! nvvideoconvert \
! capsfilter caps="video/x-raw, format=I420" \
! x264enc ! h264parse ! filesink location=output.mp4

This is the simplified version of the main pipeline. Expected to output the output.mp4 file but it prints out these logs then hangs:

Pipeline is PREROLLING ...
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261

Your pipeline is incorrect. Please try the pipeline below:

 gst-launch-1.0 uridecodebin uri="file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4" ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 ! nvinfer config-file-path=dstest1_pgie_config.txt ! queue2 ! nvvideoconvert  ! x264enc ! h264parse ! 'video/x-h264,stream-format=byte-stream' ! filesink location=output.h264

@yuweiw
Thank you for your help, your pipeline worked.
However, for the first time running, it still hangs for about 2 minutes after this log

Pipeline is PREROLLING ...
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Redistribute latency...

I’ve tested your pipeline in DS 6.1.1 and DS 6.3. This behaviour only occurs in DS 6.3. In DS 6.1.1, after the above logs, it quickly prints below logs and proceeds without any observed hang.

Setting pipeline to PLAYING ...
New clock: GstSystemClock

For the first time running, it will generate the engine file. That will cost some time.

Hi @yuweiw,
I want to clarify that I am not referring to the time spending for generating engine file. The hanging issue happens after that.

Did you run the pipeline I attached? I run that with DS 6.3 on T4. After the above logs, it quickly prints below logs and proceeds without any observed hang.

Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

@yuweiw
Did you run the pipeline I attached?
→ yes, I did, and there is no issue if I run it on PC with GPU (RTX 2080 Ti).

The problem happens only when I run it on Jetson orin nano, since the device does not have hardware encoder, I have no choice but using x264enc.

You mean when use x264enc on Jetson orin nano, it will hang 2m on DS 6.2 and 6.3. But it runs well in DS6.1. Is that right? Could you use fakesink to narrow down the scope of this problem? You can replace the plugins after the pipeline with faksink one by one.

@yuweiw

You mean when use x264enc on Jetson orin nano, it will hang 2m on DS 6.2 and 6.3, But it runs well in DS6.1. Is that right?

→ Yes

Could you use fakesink to narrow down the scope of this problem?

→ It still hangs if I use fakesink instead of filesink.

I mean you can add the fakesink after the nvvideoconvert.

gst-launch-1.0 uridecodebin uri="file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4" ! \
m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 ! \ 
nvinfer config-file-path=dstest1_pgie_config.txt ! \
queue2 ! nvvideoconvert  ! fakesink

Since we just released the DS6.4, could you also try that on DS6.4?

Hi @yuweiw ,
Apologies for the delayed response. After upgrading Jetpack to version 6.0 and using DeepStream 6.4, it appears that the issue with x264enc has been resolved. The pipeline mentioned above runs successfully without any observed hang.

However, during further testing with deepstream-test3, the log shows following errors:

libv4l2: error getting capabilities: Inappropriate ioctl for device
and
Error: Decodebin did not pick nvidia decoder plugin.

Do you have any insights about this issue?

Could you modify the source code like below and try that?
deepstream_test_3.py

        # Set the source bin ghost pad
        bin_ghost_pad=source_bin.get_static_pad("src")
        if not bin_ghost_pad.set_target(decoder_src_pad):
                sys.stderr.write("Failed to link decoder src pad to source bin ghost pad\n")
        if features.contains("memory:NVMM"):
            print("Decodebin pick nvidia decoder plugin.\n")
        else:
            print("Decodebin did not pick nvidia decoder plugin.\n")

@yuweiw :
This is the log after modification, still error.

Decodebin child added: source

Decodebin child added: decodebin0


**PERF:  {'stream0': 0.0}

Decodebin child added: qtdemux0

Decodebin child added: multiqueue0

Decodebin child added: h264parse0

Decodebin child added: capsfilter0

Decodebin child added: aacparse0

Decodebin child added: avdec_aac0

Decodebin child added: nvv4l2decoder0

Opening in BLOCKING MODE
libv4l2: error getting capabilities: Inappropriate ioctl for device
Decodebin child added: avdec_h264-0

In cb_newpad

gstname= video/x-raw
caps= <Gst.Caps object at 0xffff59df8be0 (GstCaps at 0xffff00001990)>
features= <Gst.CapsFeatures object at 0xffff59df90c0 (GstCapsFeatures at 0xffff00008860)>
Decodebin did not pick nvidia decoder plugin.

In cb_newpad

gstname= audio/x-raw
caps= <Gst.Caps object at 0xffff59df90c0 (GstCaps at 0xfffef40022d0)>
Error: gst-stream-error-quark: Internal data stream error. (1): ../gst/isomp4/qtdemux.c(6749): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
Exiting app

I think the reason comes from this child element, but I don’t know why avdec_h264 is added while it already has nvv4l2decoder0?

Decodebin child added: avdec_h264-0

Is there any chance that it failed to use hardware decoder b/c of the error related to libv4l2, so it falls back to using avdec_h264?

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Yes. But it should be work with avdec_h264 and x264enc. Could you run that with GST_DEBUG=3?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.