Hey customer, you need to change some config items to run the yolov4 engine file with dynamic batch size, for config.pbtxt, you need apply following:
root@p4station:/home/bcao/customer-bug/triton-bs# diff config.pbtxt config.pbtxt.fix
6c6,7
< max_batch_size: 4
---
> max_batch_size: 0
>
8,11d8
< dynamic_batching {
< preferred_batch_size: [ 1, 4 ]
< max_queue_delay_microseconds: 100
< }
16,17c13
< format: FORMAT_NCHW
< dims: [ 3, 608, 608 ]
---
> dims: [-1, 3, 608, 608 ]
24c20
< dims: [ 22743, 1, 4 ]
---
> dims: [-1, 22743, 1, 4 ]
29c25
< dims: [ 22743, 80 ]
---
> dims: [-1, 22743, 80 ]
And for config_infer_primary_yolov4.txt you need to change the tensor_order
root@p4station:/home/bcao/customer-bug/triton-bs# diff config_infer_primary_yolov4.txt config_infer_primary_yolov4.txt.fix
20c20
< tensor_order: TENSOR_ORDER_NONE
---
> tensor_order: TENSOR_ORDER_LINEAR
I had shared all the correct configs for you, you can find the details inside them.
config.pbtxt.fix (598 Bytes) config_infer_primary_yolov4.txt.fix (1.3 KB) source4_primary_yolov4.txt (3.9 KB)