I am training a custom network which is composed of residual connections (like in resnet block). The problem is in some case, the weights of one path will be eventually all zeros during the training and so the resnet block becomes the identity mapping. The trained network still works well with these zero-paths in float32 but fails in the int8 calibration. I have acknowledged that TensorRT is not allowed to do the calibration for layer with all output zero and it will raise this error:
[TensorRT] ERROR: Tensor ... is uniformly zero; network calibration failed.
In this case I cannot do nothing except to retrain another model and hope that there will not be any identity mapping in the network. Is there anyway that I can surpass this problem ?