Multi-stream real-time predection with jetson nano

I used gstreamer + opencv for decoding 12 multi-stream ip camera with 480p5.
The predection times of model in nano are 100ms and 130ms for batch_size=1 and batch_size=2.
I used threads for H264 HW decoding. and I want to know how to handle this problem for 12 cameras at the same time? I want to know in Nvidia demo shown 8 streams 1080p30 processed at the same time, how do they thandle this challange. I guess they don’t feed inputs of 8 stream to model at the same time, right? I think they feed the first two cameras as batch_size=2 and then feed to model and then second two cameras to model as sequentioal and so on , right? if so, assume they feed each of two cameras to model as same time and also they decode 8 streams at the same time, so if we feed first two camera into model in the time of T1, the frames of rest cameras (i.e 3-8) are rejected in the time of T1 for processing?

If you check deepstream plugin, you will find

  1. They do use batch = 8. So it’s at the same time.
  2. The model is extremely small, because it’s deeply optimized.

Thanks for your reply.
Do you know what models thay used? ssd_mobilev1/v2?

hi LoveNvidia:
the model in Demo for deepstream is Resnet10 for feature extraction, post-process is coded in nvinfer Plugin