Hello,
Why does DeepStream 6.2 require the h265parse element to be used between the elements nvv4l2h265enc and nvv4l2decoder as mentioned here Nvv4l2h265enc produces no valid frames for nvv4l2decoder whereas previous versions didn’t?
Thank you!
In GStreamer, you should always have a parser element after the encoder. The parser divides or converts a raw encoded stream into a format that can be consumed downstream with the required metadata. If it was working in the past, it means that the encoder was previously performing some operations that should be performed by a parser, or the decoder didn’t care that the stream was missing information. It can be confusing because other libraries mix encoding, parsing, and even the container into the same call, but that’s not the case with GStreamer.
Hi @miguel.taylor,
Thank you very much for your reply!
Guess I was curious to understand what were those operations that the nvv4l2h265enc was doing in previous DeepStream versions which meant it didn’t require a parser? The trouble being that the latency increases substantially after introducing the h265parse element to our DeepStream 6.2 pipeline.
To be more precise, the h265parse latency is close to the inverse of the frame rate (i.e. 17ms with 60fps, 33ms with 30fps, 67ms with 15fps, and so forth). Any ideas why?
Thank you so much!
Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
Hi @Fiona.Chen,
• Hardware Platform (Jetson / GPU) - Jetson Orin
• DeepStream Version - 6.2
• JetPack Version (valid for Jetson only) - libnvidia-container0:arm64, 0.11.0+jetpack, arm64, NVIDIA container runtime library
• TensorRT Version - not sure, using libnvidia-container1:arm64 1.13.5-1
• NVIDIA GPU Driver Version (valid for GPU only) - Na
Hope this helps. Let me know if you need any additional info.
Thank you very much for looking into this.
h265parse is needed before nvv4l2h265enc to make sure the stream is fed in NALs.
Even the older version did not check the input, the h265parse is also necessary.
Hi @Fiona.Chen,
Thank you for the reply!
The challenge is that the older version works without the parser. The extra latency of one frame we see with the new version (that requires the h265parse element) is really an issue for our use case.
Is there anything we can do to prevent it? E.g. is it possible to make the nvv4l2h265enc add marker bits to signal to the parser the end of the AU, so it doesn’t have to wait for the next frame (assuming that’s the root cause)? We’ve tried adding “insert-aud: True” but that didn’t solve the issue.
Thank you once again for your help!
How long is the latency? How did you measure the latency?
Hi @Fiona.Chen,
The h265parse latency is approximately one frame. So, about 17ms with 60fps, 33ms with 30fps, 67ms with 15fps, and 167ms with 6fps. We’ve been measuring it with our own custom profiling/logging but also with gst-shark.
Is there anything we can do to prevent it? E.g. is it possible to make the nvv4l2h265enc add marker bits to signal to the parser the end of the AU, so it doesn’t have to wait for the next frame? We’ve tried adding “insert-aud: True” but that didn’t solve the issue.
Thank you once again for your help!
What are the parameters with your nvv4l2h265enc?
Hi @Fiona.Chen,
bitrate: 10000000, preset-level: 1, insert-sps-pps: 1, and maxperf-enable: 1. Everything else is default. We’ve tried a few different params, including insert-aud: 1, but nothing fixed the issue so far.
Thank you for your help!
We are investigating this issue and will be back when there is progress.
Hi @Fiona.Chen, thank you for letting us know and for the help!
Hi @Fiona.Chen, just a quick note to say that your colleague Viranjan Pagar has provided a solution for this issue (see below). We no longer need to use the element h265parse in our pipeline. Thank you.
"Solution is:
in the gstv4l2videodec.c file please comment out below part of the code,
if ((GST_V4L2_PIXELFORMAT(obj) == V4L2_PIX_FMT_H264) ||
(GST_V4L2_PIXELFORMAT(obj) == V4L2_PIX_FMT_H265))
{
if ((GST_BUFFER_FLAG_IS_SET (GST_BUFFER_CAST(frame->input_buffer),
GST_BUFFER_FLAG_DELTA_UNIT)) &&
(self->idr_received == FALSE))
{
GST_DEBUG_OBJECT (decoder, “Delta Unit Received, Dropping…”);
gst_video_decoder_drop_frame (decoder, frame);
return GST_FLOW_OK;
}
self->idr_received = TRUE;
}
you will find this file in the sources that we release, you will have to compile it and create libgstnvvideo4linux2.so library and use it.
Hopefully we will add a fix for this issue in upcoming release, till then you can use this fix, thank you for pointing this out."
Glad to hear it!
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.