Quick uneducated questions regarding TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I just found out about TensorRT and have a few uneducated questions, I assume this is the right place to ask.

1.Do I understand correctly that the purpose of TensorRT is to speed up model performance in a production setting and not during training?
2. Are dropout layers expected to be working identically to the original (PyTorch) model? I’m performing Monte Carlo predictions in production (meaning averaging the output of several forward passes with Dropout layers enabled) and am wondering if I can expect a TensorRT model to give me “the same” results.

Thanks in advance.

4

Hi,

Answer for your both queries is Yes.
NVIDIA TensorRT is a C++ library that facilitates high performance inference on NVIDIA GPUs. It is designed to work in connection with deep learning frameworks that are commonly used for training. TensorRT focuses specifically on running an already trained network quickly and efficiently on a GPU for the purpose of generating a result; also known as inferencing.

Please refer TensorRT documentation for more details.
https://docs.nvidia.com/deeplearning/tensorrt/index.html
Thank you.