Hi all, I want to know how I should write code to perform layer fusion.
I want to modify the following sample files present in the tensorrt files to perform layer fusion
Hi SivaRamaKrishna, thank you for providing this information. I have read this link and have further clarifications.
These clarifications are w.r.t
When the code parsers the pretrained model and builds the engine
In this case it says it tries to optimize by fusing layers which are possible. This information can be available (seen) through the log information logged by the ILogger.
I am pasting the information from the document “To investigate which fusions have happened, or has not happened, the builder logs its operations to the logger object provided during construction. Optimization steps are at the kINFO log level. To see these messages, ensure you log them in the ILogger callback.” Let me know where in the code and how I should write the code to log for seeing this information?
This is the second clarification I have. Let us take an example code provided in the path /usr/src/tensorrt/samples/python/network_api_pytorch_mnist/sample.py In this file it manually creates the network with the similar layers, weights and bias taken from the pretrained model (present in the file model.py in the same directory).
I have a question on this. Does it perform layer fusion automatically in this case also?. If ‘yes’ then provide me the information about how it does?. If ‘no’ then let me know how to perform it manually by ourselves. The order of the layers present in this network are
Dear @trivedi.nagaraj,
The layer fusion happen at the time of TRT engine building step. You can use verbose flag in trtexec to get more information about engine building. Please check trtexec --help for details.
Hi SivaRamaKrishna, it is not about the trtexec. I was specifically asking about the python sample program
/usr/src/tensorrt/samples/python/network_api_pytorch_mnist/sample.py
In this file how I can make use of TRT logger to dump layer fusion information. I have already pasted the information from the TensorRT SDK manual in the previous post. I am again pasting it here “To investigate which fusions have happened, or has not happened, the builder logs its operations to the logger object provided during construction. Optimization steps are at the kINFO log level. To see these messages, ensure you log them in the ILogger callback.”
My query is how I can make use of the logger to get layer fusion information. How and from where the logger gets this information about the builder that it has fused few layers, particularly for the programs present in the python sample directory
/usr/src/tensorrt/samples/python/network_api_pytorch_mnist/