I’ve been using the method described in the article below in order to run our network in INT8 instead of FP16. The speedup is really cool, and the visual results (i.e. after I process the network and visualize whats needed) seems to be ok.
However when I start comparing the numerical results between the FP16 and INT8 networks, I see big differences. It seems that the ratio in the numbers is correct, i.e., if the FP16 results contain a sequence of the following numbers, starting from the Xth position: -1.5, 0.34, 0.51, 3.4, -1.7
I’d see a similar sequence in the INT8, but somewhat shifted/scaled/??? such as: -5.2, 0.56, 4.53, -5.1
Is that reasonable? What’s this difference?
Anyone can clarify what exactly happens when the dynamic ranges are set?