MMAPI can't decode more than 8 video streams at the same time

Hi folks,

I want to decode 16 video streams and display a image that they are tiled.

Multimedia API’s sample, Backend, is similar to the purpose, so I added some touches to the sample code to be able to input more video streams.

Then, I see following error messages when it runs with 9 video streams.

libv4l2: error attempting to open more then 16 video devices
[ERROR] (NvV4l2Element.cpp:63) <dec8> Could not open device '/dev/nvhost-nvdec': Device or resource busy
[ERROR] (NvV4l2ElementPlane.cpp:412) <dec8> Output Plane:Error in VIDIOC_REQBUFS at output plane: Bad file descriptor
[ERROR] (NvV4l2ElementPlane.cpp:412) <dec8> Capture Plane:Error in VIDIOC_REQBUFS at output plane: Bad file descriptor 
Could not create decoder

In sample code, one NvVideoDecoder and one NvVideoConverter are used for each video streams. So a decoder for 9th video stream could not be created because it had been using 2 * 8 = 16 video devices already. I think this is reason for the error.
Is my understanding correct?

If it is correct, how can I achieve my purpose?

p.s.
I tried GStreamer to decode 16 video streams, and it worked. But there are performance problems. So now I’m trying to use MMAPI.
And I also wonder why GStreamer(OpenMax) doesn’t have the restrictions that MMAPI(V4L2) has.

Hi KentaroH,
Simultaneous 16 video decoding is over spec. you will also hit performance issue in using MM APIs. What is the resolution in your case?

Here is a case of 4 1080p25 decoding:
https://devtalk.nvidia.com/default/topic/1014789/jetson-tx1/-the-cpu-usage-cannot-down-use-cuda-decode-/post/5189145/#5189145

Do you hit the error in running like

./video_decode ~/a.h264 H264 & ./video_decode ~/a.h264 H264 -ww 360 -wh 180

Is it r24.2.1 or r28.1?

Hi DaneLLL,

I hope to decode 1 to 16 video streams whose total resolution is 4K@30 * 2.
For Example, when input streams are 720p@30, max.number of streams is 16. And when input streams are 1080p@30, max.number of streams is 8.
However, a inputing 16 streams is mandatory but the spec of the total resolution is able to change according to actual performance.

So I’m trying to check actually how many 1080p@30 video streams can be decoded without frame drops.
For now, MMAPI seems to have a better performance to decode and display streams than GStreamer.
But if I can not avoid the 16 video devices restriction, I have to find other way.

Do you hit the error in running like
./video_decode ~/a.h264 H264 & ./video_decode ~/a.h264 H264 -ww 360 -wh 180
There is not the error. I also run 9 video_decode, and there is no error too.
I checked the source code and found that it creates only a decoder if it runs without the file output option.
Does it mean that I can’t use a converter if I want to decode 16 video streams?
Or is there any way to avoid the 16 video devices restriction?

Is it r24.2.1 or r28.1?
It’s R28.1.

Hi KentaroH,
It is a limitation from 3rd party code

#define V4L1_MAX_DEVICES 16

So you will attach the 16 720p video frames into one frame? What is the reason you cannot run each input stream into individual process?

Hi DaneLLL,

It is a limitation from 3rd party code
Thanks for the useful information.
I modified the libv4l2.so, and I was able to display 9 input video streams!

But when I tried to input 16 video streams, I got a new error.

libv4l2_nvvidconv: Error: MAX number of NVVIDCONV instances (10) already created.

Do you have any ideas to avoid it?

So you will attach the 16 720p video frames into one frame?
For now, I have no plan to do it.
Incidentally, if I use GStreamer, I need to do it. Because I’d like to use nvoverlaysink for good rendering performance and nvoverlaysink have only one sink pad.

Just conjecture, perhaps you could write software to make an eight streams look like it came from a single stream…do this twice in separate threads, and then feed both to the same application to put those together…a tree of streams. I don’t know how the GPU would feel about it :)

Hi KentaroH,
The limitation of max instances has been removed in r28.1.

libv4l2_nvvidconv: Error: MAX number of NVVIDCONV instances (10) already created.

Could you double confirm your release version?
$ head -1 /etc/nv_tegra_release

Hi DaneLLL,

You are right.
The result of my comment #5 came from a R24.2.1 borad.
I mixed up the version. Sorry, I hadn’t checked thoroughly.

I retried by using R28.1 one, and finally I could display 16 input streams without such error.

Thanks for your kindly support.