Error Code 2: OutOfMemory Error during INT8 calibration

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I get the “[virtualMemoryBuffer.cpp::nvinfer1::stdVirtualMemoryBufferImpl::resizePhysical::161] Erro Code 2: OutOfMemory (no further information)” error during dynamic input tensor shape int8 calibration from time to time. I monitor the GPU and RAM memory consumption and there is always free memory during the process.

The calibration process outputs this message around 40 times just before it finishes but the quantized model works well.

Do you know what the problem could be ?

Thanks!

A clear and concise description of the bug or issue.

Environment

TensorRT Version: 8.4.0.6
GPU Type: GeForce RTX 2080 Super
Nvidia Driver Version:
CUDA Version: 11.6.1
CUDNN Version: 8.3.2.44
Operating System + Version: Win 10
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi, Please refer to the below links to perform inference in INT8

Thanks!

4

I appreciate your reply, NVES, but I couldn’t find any online documentation or article covering the issue I described.

My int8 calibration procedure works well for all of my models but the one that causes this error to happen and I couldn’t find the culprit.

I’d appreciate if someone could at least explain what kind of problem this message refers to. Insufficient RAM, GPU VRAM… ?

Thanks!

5

Hi,

This happens during tactic selection, It’s more related to GPU Memory, so as long as your network generates and works properly, then there is nothing to worry about. You may try to increase the workspace.

Thanks.

1 Like
6

Hi spolisetty,

Thanks so much for the reply. It is very appreciated.

Awesome support from the NVIDIA’s staff as always :)

Have a great day!

1 Like
7

Thank you :)

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.