Out of memory error when trying to convert model using INT8 precision mode

Hi,

I’ve been following your example to speed up inference with TensorRT, but when I try conversion using INT8 precision mode my program is killed during .build() call.

Output of sudo dmesg -T is:

[Thu Sep 14 04:40:02 2023] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-54.scope,task=python,pid=218008,uid=1000
[Thu Sep 14 04:40:02 2023] Out of memory: Killed process 218008 (python) total-vm:87797408kB, anon-rss:0kB, file-rss:1177636kB, shmem-rss:0kB, UID:1000 pgtables:38348kB oom_score_adj:0
[Thu Sep 14 04:40:07 2023] oom_reaper: reaped process 218008 (python), now anon-rss:0kB, file-rss:1179088kB, shmem-rss:0kB

Moreover, whenever I use Tensorflow, my memory gets used up almost completely, is this normal? Here’s the output of jtop:

My code:

import sys
import os
import cv2
import numpy as np
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
from pose_estimation.pose_net import PoseNet

POSE_SAVED_MODEL_DIR="./weights/pose_native_saved_model"

def calibration_input():
    for i in range(100):
        batched_input = np.random.random((1, 224, 224, 3)).astype(np.float32)
        batched_input = tf.constant(batched_input)
        yield (batched_input,)

# Instantiate the TF-TRT converter
converter = trt.TrtGraphConverterV2(
   input_saved_model_dir=POSE_SAVED_MODEL_DIR,
   use_dynamic_shape=True,
   dynamic_shape_profile_strategy='Optimal',
   max_workspace_size_bytes=4000000000,
   precision_mode=trt.TrtPrecisionMode.INT8
)
 
# Convert the model into TRT compatible segments
trt_func = converter.convert(calibration_input_fn=calibration_input)
converter.summary()

POSE_TRT_MODEL_DIR="./weights/pose_trt_09.14_01_int8"

def input_fn():
   batched_input = np.random.random((1, 224, 224, 3)).astype(np.float32)
   batched_input = tf.constant(batched_input)
   yield batched_input

converter.build(input_fn=input_fn)
converter.save(output_saved_model_dir=POSE_TRT_MODEL_DIR)

Thanks for sharing your issue.
Our internal team will check and update more info with you.

Dear @nikola2,
May I know the Jetpack version?

• Hardware Platform Jetson
• DeepStream Version 6.3
• JetPack Version 5.1
• TensorRT Version 5.1

Dear @nikola2,
From jtop I see, it is Jetson AGX Orin. Is it Jeston AGX Orin or AGX Orin NX ? Could you please also share the model for reproducing the issue?

It’s Jeston AGX Orin
Sure, here’s the link

Dear @nikola2 ,
I notice similar behavior as yours on my machine.

1 Like

Dear @nikola2,
Did you check if TF → ONNX → TRT working with out OOM(out of memory) issue?

1 Like

Hi, I switched to DeepStream and TRT and now my application has no OOM issue.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.