# TensortRT Memory Utilization

**URL:** <https://forums.developer.nvidia.com/t/tensortrt-memory-utilization/147388>\
**Category:** TensorRT\
**Created:** [August 18, 2020, 7:43pm UTC](https://forums.developer.nvidia.com/t/tensortrt-memory-utilization/147388 "2020-08-18T19:43:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![whanafy](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@whanafy](https://forums.developer.nvidia.com/u/whanafy)\
**Post date:** [August 18, 2020, 7:43pm UTC](https://forums.developer.nvidia.com/t/tensortrt-memory-utilization/147388/1 "2020-08-18T19:43:49Z")

</div>

## Description

I am using TensortRT on Nvidia-1080 GPU. I was trying to find out the memory usage of the inference engine on the GPU.  
I was trying to find this using the approach described in [Overview — NVIDIA TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#faq) which roughly says the memory usage is `The Serialized Engine + 2 * Bindings Sizes + Device Engine Memory Size`. However, I found that the actual allocated GPU memory by the whole process is quite different.

The memory consumption computed by the nvidia-smi command `nvidia-smi --query-compute-apps=pid,used_memory --format=csv` is not the same, to be exact it is **270 MB** greater (this is the same value when computed across different engines).

I don’t know why there is a difference in the values, or what is missing in this memory computation?

---

<div class="post-metadata">

**Author:** ![AakankshaS](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aakankshas/32/14047_2.png) [@AakankshaS](https://forums.developer.nvidia.com/u/AakankshaS)\
**Post date:** [August 19, 2020, 7:01am UTC](https://forums.developer.nvidia.com/t/tensortrt-memory-utilization/147388/2 "2020-08-19T07:01:33Z")

</div>

Hi @whanafy,  
Please refer to the below link for FAQ section of the TRT document.

> **[Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#faq)**
>
> This is the revision history of the NVIDIA TensorRT 8.4 Developer Guide.

**Note:** The CUDA infrastructure and device code also consume device memory. The amount of memory will vary by platform, device, and TensorRT version. Use cudaGetMemInfo to determine the total amount of device memory in use.  
Thanks!