Jetson video hardware engine core dumps when the number of parallel pipelines is more than 6

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Orin-nano
• DeepStream Version DS v7.1
• JetPack Version (valid for Jetson only) Just Tested on Jetson
• TensorRT Version 8.6.2.3-1+cuda12.2
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? Run the given pipeline in more than 6 different threads.
• Requirement details

In an application we need to read RTSP streams (H264) of more than 100 cameras and decode their streams every 1 second in a 200 ms time window. When we use GStreamer software decoders like avdec_h264, the CPU reaches its limits and gives a lot of latency (more than 1 second for 10 cameras), and most streams miss frames for a relatively long time period (sometimes 1-2 seconds).

pipeline_str = g_strdup_printf("rtspsrc location=%s name=rtsp protocols=tcp drop-on-latency=false latency=50 ! rtph264depay name=rtpdepay request-keyframe=true wait-for-keyframe=true ! h264parse ! avdec_h264 ! valve drop=true name=valve ! videoconvert ! video/x-raw,format=RGB ! pngenc compression-level=0 ! appsink name=appsink sync=true emit-signals=false drop=true max-buffers=1", URL);

When we use the deepstream component as below, it reduces this time incredibly to less than 15 ms, but it gives us a core dump error for more than 6 cameras.

pipeline_str = g_strdup_printf("rtspsrc location=%s name=rtsp protocols=tcp drop-on-latency=false latency=50 ! rtph264depay name=rtpdepay request-keyframe=true wait-for-keyframe=false ! h264parse  ! nvv4l2decoder ! nvvideoconvert ! video/x-raw, format=(string)I420 ! nvjpegenc quality=100 ! appsink name=appsink sync=true emit-signals=false drop=true max-buffers=1", URL);

I think, since the DeepStream components are using the dedicated hardware nvdec, it reaches its limits.

Q1: is there any component for nvv4l2decoder, and nvjpegenc to do the decoding batch by batch? for example, 10 input streams at a time?

Q2: is it possible to use NVDEC library to create a gstreamer plugin to do this (below image).

Please click here to see the image

For example, suppose we have 100 cameras. The custom plugin (based on NVDEC) will process 5 camera streams at a time. It gets 5 buffers from each h264parse at a time, decodes it, extracts the frame, encodes it to jpg, and then starts on the next batch.

2

This table shows the hardware decoding capabilities of Orin Nano.

Currently, this is not supported. In fact, in the user driver of the hardware decoder, each decoder instance is a session, which means that it is already a batch-like processing mode.

If you want to support more decoder instances simultaneously, you may want to consider dGPU or more Jetson devices.

4

Thank you for your answer, since we plan to support more than 100 cameras, I think using a dGPU will not still help us reach such a goal. and using multiple dGPU will be also an expensive solution.

Since it is a session, if we use the lower level NVDEC, like the example inside the below link, we would be able to implement this component, get a batch of streams (from different RTSP cameras) and decode it in a session or multiple sessions. do you think we are on the right path?

We are also working on the network bandwidth problem, so assume that is not a problem right now.

5

Sorry for the long delay,

I think this may not be possible, there are quite a few reference frames cached in the decoder when decoding, and these reference frames will be lost when trying to switch decoding instances

6

Thanks again, I have another question.

In Jetson AGX Orin Specs, it says, the device has 16x 1080p30 (H.265) Video Decoder. But we hit the limit with 5 parallel pipelines I mentioned.

Do nvvideoconvert, nvvideoconvert, and nvjpegenc in one pipeline all use separate sessions? So 5 pipelines, each with 3 sessions, and we hit the limit?

7

In the above spec，if you use Orin nano, can decode 11-channels 1080@30fps or 5-channel 1080@60fps streams， Or other combinations in the table

16x 1080p30 (H.265) seems to be the encoding capability of the AGX Orin 64G

Orin Nano 8G

1x 4K60 (H.265)
2x 4K30 (H.265)
5x 1080p60 (H.265)
11x 1080p30 (H.265)
8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.