Inference from the TensorRT engine

Please provide complete information as applicable to your setup.

*NVIDIA Jetson Xavier NX
DeepStream Version5.0
Jetpack 4.4.1 [L4T 32.4.4]
CUDA: 10.2.89
OpenCV: 4.1.1 compiled CUDA: NO
VPI: 0.4.4
Vulkan: 1.2.70

I am developing a face detection app based on deepstream sample 2, using keras facenet model.
I successfully managed to convert Keras Model to TensorRT Using ONNX. i have .pb ,.onnx and .plan file.

from PIL import Image
import numpy as np
from mtcnn.mtcnn import MTCNN

import engine as eng
import inference as inf
import keras
import tensorrt as trt 
from utils import *

TRT_LOGGER = trt.Logger(trt.Logger.INTERNAL_ERROR)
trt_runtime = trt.Runtime(TRT_LOGGER)

engine_path = "/home/blaise/Desktop/Facedetection/models/facenet.plan"
input_file_path = 'blaise.png'
dataset_embeddings_path = '/home/blaise/Desktop/Facedetection/models//embeddings-yasser.npz'
HEIGHT = 160
WIDTH = 160

# load dataset embeddings
dataset_embeddings = np.load(dataset_embeddings_path)
faces_embeddings, labels = dataset_embeddings['arr_0'], dataset_embeddings['arr_1']
# Normalize dataset embeddings
faces_embeddings = normalize_vectors(faces_embeddings)

detector = MTCNN()

face_array = extract_face_from_image(input_file_path, detector)

face_pixels = face_array
# scale pixel values
face_pixels = face_pixels.astype('float32')
# standardize pixel values across channels (global)
mean, std = face_pixels.mean(), face_pixels.std()
face_pixels = (face_pixels - mean) / std
# transform face into one sample
samples = np.expand_dims(face_pixels, axis=0)
# make prediction to get embedding

engine = eng.load_engine(trt_runtime, engine_path)

h_input, d_input, h_output, d_output, stream = inf.allocate_buffers(engine, 1, trt.float32)
yhat = inf.do_inference(engine, samples, h_input, d_input, h_output, d_output, stream, 1, HEIGHT, WIDTH)


face_to_predict_embedding = normalize_vectors(yhat)
result = predict_using_min_l2_distance(faces_embeddings, labels, face_to_predict_embedding)

print('Predicted name: %s' % (str(result).title()))

When i use this script above to for inference from the TensorRT engine using real image, i get this error

ImportError: /usr/lib/aarch64-linux-gnu/ cannot allocate memory in static TLS block

How can i solve this problem?

After looking online, they are suggestiong to use the code below:
export LD_PRELOAD = /usr/lib/aarch64-linux-gnu/
But i am still getting the same error.

Hey, the issue seems not related to Deepstream SDK.
Can you run your model using trtexec successfully?

I haven’t. How do i do that?

Refer TensorRT/ at master · NVIDIA/TensorRT · GitHub