Why tensorRT occupy many memory ?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
2

Hi,

There is two source consumes memory:
1. Loading libraries: (TensorRT, cuDNN, cuBLAS…)

  • Amount: around 600Mib (TensorRT3)
  • Required but is shared with all the processes.

2. Building inference engine:

  • Amount: depends on the network size
  • Can be limited by setMaxWorkspaceSize() and setMaxBatchSize(). Each process has their own consumption.

Thanks.