Deepstream batch mode and non batch mode performance

Please provide complete information as applicable to your setup.

**• AGX ( 32G )
**• DS5.1
**• Jetpack 4.5.1
**• TensorRT 7.1.3
**• Nvidia L4T 32.5.1
• Issue Type( questions, new requirements, bugs)

I have testing the DeepStream pipeline on batch mode and multiples deepstream instance mode .

Method-1 :

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://…/…/streams/sample_1080p_h264.mp4
num-sources=24

[primary-gie]
enable=1
model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b30_int8.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=24

  1. Set number of streams to 24 in the deepstream config file [source0]
  2. Set batch size to 24 in the [primart-gie ]
  3. Run DeepStream application with file-loop enabled

Method-2

  1. Set number of streams to 1 in the deepstream config file [source0]
  2. Set batch size to 1 in the primary-gie
  3. Instantiate 24 individual deepstream instances

Observation: The first method delivers almost twice FPS performance at sink. Is this because the first method takes advantages of parallel execution within the GPU for frames coming from a single muxer (sits between inference engine and decoder), whereas the second method uses preemptive scheduling of GPU for frames coming from 24 individual muxer?

Sorry for the late.

  1. Instantiate 24 individual deepstream instances
    ->24 individual processes or 24 individual pipelines within one process?
    also can you send the config files for both the cases?

hi , I mean 24 instances . below are the config file on batch mode for deepstream , pgie ( using tiny yolo2,FP16 ) and sgie ( resnet50,INT8 ) ,
while multiples instance ( deepstream_nonbatch.txt ) and pgie_non_batch.txt
deepstream_app_config.txt (1.7 KB)
deepstream_non_batch.txt (1.7 KB)
pgie.txt (672 Bytes)
pgie_non_batch.txt (671 Bytes)
sgie.txt (3.3 KB)

Please refer to Developer Guide :: NVIDIA Deep Learning TensorRT Documentation