I have a primary-gie and a secondary-gie for inference so when multiple primary instances were found, the same amount of secondary inference were triggered. As a result, the secondary-gie took significantly more time than the primary-gie even though the secondary-gie utilizes a smaller model.
I tried creating a secondary model with batch-size 2 but it ended up producing very bad results. So how can I improve the latency of secondary-gie?
Sure. We use DS for traffic light recognition, i.e the primary-gie for detection, and the secondary-gie for getting the sematic information of each traffic light detected. And when multiple traffic lights were detected, the secondary-gie would consume significantly more time.