Nvv4l2h265enc hangs when encoding high resolution, high bitrate video

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
NVIDIA GeForce RTX 3060 Laptop GPU
• DeepStream Version
7.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
550.54.15
• Issue Type( questions, new requirements, bugs)
bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Run the following pipeline:

docker run \
    -it --rm \
    --name test \
    --entrypoint gst-launch-1.0 \
    --gpus all \
    -e GST_DEBUG=2,GST_SCHEDULING:5,v4l2videoenc:9,nvv4l2h265enc:9 \
    -e GST_DEBUG_NO_COLOR=1 \
    nvcr.io/nvidia/deepstream:7.0-samples-multiarch \
    uridecodebin uri="https://eu-central-1.linodeobjects.com/savant-data/demo/Free_City_Street_Footage.mp4" ! \
    nvvideoconvert ! \
    'video/x-raw(memory:NVMM),width=7680,height=4320' ! \
    nvv4l2h265enc bitrate=200000000 ! \
    fakesink async=false sync=false qos=false enable-last-sample=false

The pipeline upscales 25 fps video to 8k.

nvv4l2h265enc hangs after processing few frames:

0:00:05.278233144     1 0x74d84c0014d0 DEBUG         GST_SCHEDULING gstpad.c:4449:gst_pad_chain_data_unchecked:<fakesink0:sink> called chainfunction &gst_base_sink_chain with buffer 0x74d7f0000be0, returned ok
0:00:05.278246910     1 0x74d84c0014d0 LOG             v4l2videoenc gstv4l2videoenc.c:1478:gst_v4l2_video_enc_loop:<nvv4l2h265enc0> Allocate output buffer
0:00:05.278258081     1 0x74d84c0014d0 LOG             v4l2videoenc gstv4l2videoenc.c:1509:gst_v4l2_video_enc_loop:<nvv4l2h265enc0> Process output buffer
0:00:05.279577047     1 0x74d84c001270 DEBUG         GST_SCHEDULING gstpad.c:4443:gst_pad_chain_data_unchecked:<capsfilter0:sink> calling chainfunction &gst_base_transform_chain with buffer buffer: 0x74d804005cc0, pts 0:00:01.320000000, dts 99:99:99.999999999, dur 0:00:00.040000000, size 64, offset none, offset_end none, flags 0x200
0:00:05.279590903     1 0x74d84c001270 DEBUG         GST_SCHEDULING gstpad.c:4443:gst_pad_chain_data_unchecked:<nvv4l2h265enc0:sink> calling chainfunction &gst_video_encoder_chain with buffer buffer: 0x74d804005cc0, pts 0:00:01.320000000, dts 99:99:99.999999999, dur 0:00:00.040000000, size 64, offset none, offset_end none, flags 0x200
0:00:05.279598297     1 0x74d84c001270 DEBUG           v4l2videoenc gstv4l2videoenc.c:1639:gst_v4l2_video_enc_handle_frame:<nvv4l2h265enc0> Handling frame 33
0:00:05.279604248     1 0x74d84c001270 DEBUG           v4l2videoenc gstv4l2videoenc.c:1710:gst_v4l2_video_enc_handle_frame: NO META RETRIEVED BY ENCODER

After stopping the pipeline, nvv4l2h265enc shows an error Failed to process frame. Maybe be due to not enough memory or failing driver:

^Chandling interrupt. (0.8 %)
Interrupt: Stopping pipeline ...
Execution ended after 0:02:07.212626114
Setting pipeline to NULL ...
0:02:09.605189276     1 0x74d84c0014d0 DEBUG           v4l2videoenc gstv4l2videoenc.c:1604:gst_v4l2_video_enc_loop:<nvv4l2h265enc0> Leaving output thread
0:02:09.605196669     1 0x74d84c001270 WARN            v4l2videoenc gstv4l2videoenc.c:1768:gst_v4l2_video_enc_handle_frame:<nvv4l2h265enc0> error: Failed to process frame.
0:02:09.605253737     1 0x74d84c001270 WARN            v4l2videoenc gstv4l2videoenc.c:1768:gst_v4l2_video_enc_handle_frame:<nvv4l2h265enc0> error: Maybe be due to not enough memory or failing driver
ERROR: from element /GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0: Failed to process frame.
Additional debug info:
gstv4l2videoenc.c(1768): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0:
Maybe be due to not enough memory or failing driver
An error happened while waiting for EOS

When changing bitrate to 100000000, the pipeline works normally. The pipeline also works with bitrate=200000000 when framerate is 50/1 (set with capssetter 'caps=video/x-raw(memory:NVMM),framerate=50/1' before nvv4l2h265enc.

nvv4l2h265enc should either report an error immediately or support encoding with those parameters.

Is there a memory limit per one frame?

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

3

Yes. You can check the video-encode-and-decode-gpu-support-matrix-new and the video-codec-sdk to check the limit.

4

Dear support. It looks like you replied formally and did not really appreciate the problem. We can encode 8K on many NVIDIA GPU models in HEVC codec, it works. However, when we increase the bitrate greater than the certain value, which is FPS-dependant, DeepStream stucks.

I really doubt that the video-encode-and-decode-gpu-support-matrix-new includes such information because it is more about codecs and encoders/decoders but does not mention any quantitative parameters.

Also, I feel that your advice related to the video-codec-sdk is too vague and formal rather than helpful. Nevertheless, I went through the document using the link and found nothing about bitrate limitations related to 8K resolution.

Could you please forward our question to a person who is in charge for the problem to investigate it thoroughly?

All the best!

5

You can get the FAMILY and NVENC Generation from the 1st link I attached, then you can find the maximum support limit from the 2nd link.

I summarized your issue in detail.

1. 8K 25fps bitrate:200000000   report memory issue
2. 8K 25fps bitrate:100000000   works normally
3. 8K  bitrate:200000000  50fps(set by the caps) works normally

There are a few things that need to be confirmed.

  1. Is 25fps the frame rate of your video source?
  2. If you set 25 fps in the capsfilter, can it work normally?
  3. Could you provide the source to us or use the videotestsrc to reproduce your problem?

We need to reproduce the problem you mentioned before we can analyze it further.

6

You can get the FAMILY and NVENC Generation from the 1st link I attached, then you can find the maximum support limit from the 2nd link.

GeForce RTX 3060 Laptop; family: Ampere, 7th Gen. It supports encoding up to 8192x8192.

  1. Is 25fps the frame rate of your video source?

Yes.

  1. If you set 25 fps in the capsfilter, can it work normally?

Works normally with bitrate=100000000, hangs with bitrate=200000000.

  1. Could you provide the source to us or use the videotestsrc to reproduce your problem?

I’m testing on this video file: https://eu-central-1.linodeobjects.com/savant-data/demo/Free_City_Street_Footage.mp4

A pipeline with videotestsrc:

docker run \
    -it --rm \
    --name test \
    --entrypoint gst-launch-1.0 \
    -e GST_DEBUG=2,GST_SCHEDULING:5 \
    --gpus all \
    nvcr.io/nvidia/deepstream:7.0-samples-multiarch \
    videotestsrc pattern=colors ! \
    'video/x-raw,width=1280,height=720,framerate=25/1' ! \
    nvvideoconvert ! \
    'video/x-raw(memory:NVMM),width=7680,height=4320' ! \
    nvv4l2h265enc bitrate=200000000 ! \
    fakesink async=false sync=false qos=false enable-last-sample=false

framerate=25/1 hangs, framerate=50/1 works.

7

We will fix this issue in the next release, but for now you can use other resolutions or frame rates.

1 Like
8

Thank you for confirming and carefully investigating the issue. We are waiting for an update. What DeepStream versions will receive such a fix?

9

We will not disclose the specific version number and dates of the unreleased version, so please stay tuned for our next release. Thanks

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.