Does network pruning speed up inference speed?

[spolisetty]

Also following may help you,

Should pruning a model prior to converting it to tensorRT make inference faster?

Hi, Sorry for the late update. This post is somehow missing in our process. 1. In general, TensorRT inference time depends on how much kernel you have launched and how long it takes. If most of the weights are pruned to 0, it should lead to much lower kernel jobs and faster. Is it possible that you are still using the old model? Please noticed that you will need to recreate the TensorRT engine if you have serialized it. 2. You can try to remove all the non-necessary node. Ex. the auxilia…