How many streams can be decoded when the GPU is running AI models?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello, I wanted to follow up regarding my previous post here.

I am running DeepStream on many cameras, and I would like to estimate how many cameras I can safely process.

I am using an NVIDIA RTX A2000 12GB. According to your previous post, I can expect to be able to decode half of the H265 streams that can be decoded by an A10. According to data from Nvidia can be found [here] (Video Codec SDK | NVIDIA Developer), I would be able to decode 162 / 2 = 81 1080 30p H265 streams.

I would like to understand if running a DeepStream pipeline (AI models) on the GPU affect the number of cameras that can be supported. Should I expect the GPU to be able to decode a lower number of cameras?

Are there any components of the GPU that are shared between the NVDEC chip and other parts of the GPU used for inference leading to potential conflicts on resources and therefore reducing the resources available to decode video streams?

Thank you

3

Hi,

The NVDEC units are completely separate and shouldn’t directly affect inference. However, other factors such as memory bandwidth, compute overhead, or context switching can impact performance, especially if you’re decoding the maximum number of streams.

4

The camera number depends on your DeepStream pipeline. Hardware decoder is not the only factor to affect the pipeline performance. E.G. When you set up an inference pipeline, the video should be processed(scaling, format conversion, dewarping …etc) to the format which the model can accept, so the GPU will be used to process the video and do the inferencing. The model’s output should be processed to the data the user want(E.G. drawing the bboxes on the video). The ethernet bandwidth will also affect the performance when you use network streams such as RTSP, HTTP,… as the input sources.

So the performance of the DeepStream pipeline is decided by all the components used in the pipeline. The hardware video decoder is just a part of them.

5

Currently I have a pipeline that can decode 64 720p H264 streams using the CPU. I want to run the same pipeline on H265 streams and decoding using NVDEC.

The pipeline runs fine with 64 720p H265 streams decoded with NVDEC. The output for nvidia-smi in this second case is:

# gpu    pwr  gtemp  mtemp     sm    mem    enc    dec    jpg    ofa   mclk   pclk 
# Idx      W      C      C      %      %      %      %      %      %    MHz    MHz 
    0    140     80      -     59     48      0     33      0      0   6801   1875 
    0     71     77      -     56     45      0     31      0      0   6801   1890 
    0    133     80      -      2      3      0     31      0      0   6801   1860 
    0    150     82      -     39     34      0     31      0      0   6801   1800 
    0    136     81      -     59     50      0     32      0      0   6801   1830 
    0    124     81      -     56     47      0     31      0      0   6801   1830 
    0    148     79      -     58     49      0     32      0      0   6801   1890 
    0     70     77      -     52     45      0     31      0      0   6801   1890 
    0    129     80      -      2      3      0     31      0      0   6801   1890 
    0    129     81      -     43     37      0     32      0      0   6801   1845 
    0    155     79      -     58     49      0     34      0      0   6801   1890 
    0    139     80      -     51     43      0     29      0      0   6801   1890 
    0    122     82      -     58     49      0     34      0      0   6801   1815 
    0     86     78      -     55     46      0     31      0      0   6801   1890 
    0    103     81      -      6      6      0     30      0      0   6801   1785 
    0    101     81      -     55     47      0     35      0      0   6801   1815 
    0    150     82      -     58     49      0     31      0      0   6801   1815 
    0    150     79      -     55     47      0     31      0      0   6801   1875 
    0    114     81      -     53     45      0     33      0      0   6801   1875

The pipeline is slowing down when decoding 64 1080p H265 streams using NVDEC. The output for nvidia-smi in this case is:

# gpu    pwr  gtemp  mtemp     sm    mem    enc    dec    jpg    ofa   mclk   pclk 
# Idx      W      C      C      %      %      %      %      %      %    MHz    MHz 
    0     65     53      -    100     12      0     36      0      0   6801   1890 
    0     65     53      -    100     10      0     48      0      0   6801   1890 
    0     69     53      -    100     13      0     56      0      0   6801   1890 
    0     85     53      -    100     11      0     42      0      0   6801   1890 
    0     80     53      -    100     12      0     44      0      0   6801   1890 
    0     71     53      -    100     11      0     42      0      0   6801   1890 
    0     63     53      -    100     12      0     44      0      0   6801   1890 
    0     63     53      -    100     11      0     50      0      0   6801   1890 
    0     85     53      -    100     13      0     53      0      0   6801   1890 
    0     70     53      -    100     12      0     35      0      0   6801   1890 
    0     64     53      -    100     12      0     46      0      0   6801   1890 
    0     72     53      -    100     12      0     50      0      0   6801   1890 
    0     63     53      -    100     14      0     41      0      0   6801   1890

We can see the sm column being at 100% when using 1080p streams. However, at the same time, the pipeline is running slower - it’s processing less frames per second. Does video decoding on the GPU use streaming multiprocessors? I wasn’t expecting this to happen since my understanding is that NVDEC is a dedicated component.

6

The “sm” is the GPU usage, the “dec” is the video decoder(NVDEC) usage, they are different hardware components.

The nvidia-smi log shows that your pipeline have some tasks which use the GPU fully. Please check your pipeline.

7

So decoding the video using the GPU should not affect the sm usage? I am surprised because the only thing that changed between the two tests was the resolution of the video streams.

8

No. It will not.

So please check your pipeline, the extra video scaling needs GPU calculating.