Out of memory error when trying to convert model using INT8 precision mode


I’ve been following your example to speed up inference with TensorRT, but when I try conversion using INT8 precision mode my program is killed during .build() call.

Output of sudo dmesg -T is:

[Thu Sep 14 04:40:02 2023] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-54.scope,task=python,pid=218008,uid=1000
[Thu Sep 14 04:40:02 2023] Out of memory: Killed process 218008 (python) total-vm:87797408kB, anon-rss:0kB, file-rss:1177636kB, shmem-rss:0kB, UID:1000 pgtables:38348kB oom_score_adj:0
[Thu Sep 14 04:40:07 2023] oom_reaper: reaped process 218008 (python), now anon-rss:0kB, file-rss:1179088kB, shmem-rss:0kB

Moreover, whenever I use Tensorflow, my memory gets used up almost completely, is this normal? Here’s the output of jtop:

My code:

import sys
import os
import cv2
import numpy as np
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
from pose_estimation.pose_net import PoseNet


def calibration_input():
    for i in range(100):
        batched_input = np.random.random((1, 224, 224, 3)).astype(np.float32)
        batched_input = tf.constant(batched_input)
        yield (batched_input,)

# Instantiate the TF-TRT converter
converter = trt.TrtGraphConverterV2(
# Convert the model into TRT compatible segments
trt_func = converter.convert(calibration_input_fn=calibration_input)


def input_fn():
   batched_input = np.random.random((1, 224, 224, 3)).astype(np.float32)
   batched_input = tf.constant(batched_input)
   yield batched_input


Thanks for sharing your issue.
Our internal team will check and update more info with you.

Dear @nikola2,
May I know the Jetpack version?

• Hardware Platform Jetson
• DeepStream Version 6.3
• JetPack Version 5.1
• TensorRT Version 5.1

Dear @nikola2,
From jtop I see, it is Jetson AGX Orin. Is it Jeston AGX Orin or AGX Orin NX ? Could you please also share the model for reproducing the issue?

It’s Jeston AGX Orin
Sure, here’s the link

Dear @nikola2 ,
I notice similar behavior as yours on my machine.

1 Like

Dear @nikola2,
Did you check if TF → ONNX → TRT working with out OOM(out of memory) issue?

1 Like

Hi, I switched to DeepStream and TRT and now my application has no OOM issue.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.