I am noticing something strange when I am inferencing from a TensorsRT graph. As I inference more frames in series the overall time per frame reduces. The data is as follows,
1frame- 6sec --0.1FPS
30frames- 6sec --5FPS
100frames- 7.25sec --13.7FPS
1000frames- 31.337sec --32FPS
10000frames- 175.118sec --57FPS
I have also calculated the time ignoring the first 15 inferences calls and it appears to follow the same pattern. So this rules out the time to initialize the graph for the first few inferences.
This model is a simple MobileNetV2 and running on a jetson nano 4gb.
code snippet of inference
start_time=time.time() for i in range(n_frames): output = frozen_func(get_img_tensor(i)).numpy() ans_arr.append(output) end_time=time.time() print("time taken - ",end_time-start_time)
keen to know what is happening here and do let me know if additional info is required.