Deserialize the cuda engine failed. Run deepstream retinaface

Creating Pipeline

Creating Source

Creating Video Converter

Creating nv3dsink

Playing cam /dev/video0
Adding elements to Pipeline

Linking elements in the Pipeline

Starting pipeline

ERROR: [TRT]: INVALID_ARGUMENT: getPluginCreator could not find plugin Decode_TRT version 1
ERROR: [TRT]: safeDeserializationUtils.cpp (323) - Serialization Error in load: 0 (Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
ERROR: [TRT]: INVALID_STATE: std::exception
ERROR: [TRT]: INVALID_CONFIG: Deserialize the cuda engine failed.
ERROR: Deserialize engine failed from file: /opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_python_apps/apps/pipeline_face/retinaface/retina_r50.engine
0:00:08.926466432 26354 0x35d16980 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_python_apps/apps/pipeline_face/retinaface/retina_r50.engine failed
0:00:08.926613344 26354 0x35d16980 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_python_apps/apps/pipeline_face/retinaface/retina_r50.engine failed, try rebuild
0:00:08.926647936 26354 0x35d16980 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: failed to build network since there is no model file matched.
ERROR: failed to build network.
0:00:08.927042080 26354 0x35d16980 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
0:00:08.927108576 26354 0x35d16980 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1822> [UID = 1]: build backend context failed
0:00:08.927136064 26354 0x35d16980 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1149> [UID = 1]: generate backend failed, check config file settings
0:00:08.927659776 26354 0x35d16980 WARN nvinfer gstnvinfer.cpp:812:gst_nvinfer_start: error: Failed to create NvDsInferContext instance
0:00:08.927700480 26354 0x35d16980 WARN nvinfer gstnvinfer.cpp:812:gst_nvinfer_start: error: Config file path: config_ds_retinaface.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
Error: gst-resource-error-quark: Failed to create NvDsInferContext instance (1): /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(812): gst_nvinfer_start (): /GstPipeline:pipeline0/GstNvInfer:primary-inference:
Config file path: config_ds_retinaface.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

Help me!

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
• The pipeline being used

Jetson Nano
DeepStream 5.1
JetPack 4.5.1
TensorRT 7.1.3

I have an error when I run the file to run deepstream and the file is below. Looks like an error about building model .engine

import sys
sys.path.append(‘…/’)
import gi
gi.require_version(‘Gst’, ‘1.0’)
from gi.repository import GLib, Gst
from common.is_aarch_64 import is_aarch64
from common.bus_call import bus_call
from common.FPS import GETFPS
import configparser
from custom_retinaface import parse_objects_from_tensor_meta
import pyds
import numpy as np
import math
import time

import ctypes

ctypes.cdll.LoadLibrary(‘/opt/models/retinaface/libplugin_rface.so’)

def coor_scale(input_height, input_width, output_height, output_width):
return max(output_height/input_height, output_width/input_width)

def osd_sink_pad_buffer_probe(pad,info,u_data):
global scale
frame_number=0
gst_buffer = info.get_buffer()
if not gst_buffer:
print("Unable to get GstBuffer ")
return

# Retrieve batch metadata from the gst_buffer
# Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
# C address of gst_buffer as input, which is obtained with hash(gst_buffer)
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))    
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
    try:
        # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
        # The casting is done by pyds.glist_get_nvds_frame_meta()
        # The casting also keeps ownership of the underlying memory
        # in the C code, so the Python garbage collector will leave
        # it alone.
        #frame_meta = pyds.glist_get_nvds_frame_meta(l_frame.data)
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    except StopIteration:
        break
    
    frame_number=frame_meta.frame_num
    result_landmark = []
    l_user=frame_meta.frame_user_meta_list
    while l_user is not None:
        try:
            # Note that l_user.data needs a cast to pyds.NvDsUserMeta
            # The casting is done by pyds.NvDsUserMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone
            user_meta=pyds.NvDsUserMeta.cast(l_user.data) 
        except StopIteration:
            break
        
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META): 
            try:
                tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
            except StopIteration:
                break
            
            layer = pyds.get_nvds_LayerInfo(tensor_meta, 0)
            result_boxes, result_scores, result_landmark = parse_objects_from_tensor_meta(layer)
            # print(result_landmark)
               
        try:
            l_user=l_user.next
        except StopIteration:
            break    
      
    num_rects = frame_meta.num_obj_meta
    face_count = 0
    l_obj=frame_meta.obj_meta_list
    while l_obj is not None:
        try:
            # Casting l_obj.data to pyds.NvDsObjectMeta
            obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            
        except StopIteration:
            break

        # set bbox color in rgba
        obj_meta.rect_params.border_color.set(1.0, 1.0, 1.0, 0.0)
        # set the border width in pixel
        obj_meta.rect_params.border_width=5
        obj_meta.rect_params.has_bg_color=1
        obj_meta.rect_params.bg_color.set(0.0, 0.5, 0.3, 0.4)
        face_count +=1
        #print(face_count)
        try: 
            l_obj=l_obj.next

        except StopIteration:
            break

    # Acquiring a display meta object. The memory ownership remains in
    # the C code so downstream plugins can still access it. Otherwise
    # the garbage collector will claim it when this probe function exits.

    # draw 5 landmarks for each rect
    # display_meta.num_circles = len(result_landmark) * 5
    display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
    ccount = 0
    for i in range(len(result_landmark)):
        # scale coordinates
        landmarks = result_landmark[i] * scale
        # nvosd struct can only draw MAX 16 elements once 
        # so acquire a new display meta for every face detected
        display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)   
        display_meta.num_circles = 5
        ccount = 0
        for j in range(5):
            py_nvosd_circle_params = display_meta.circle_params[ccount]
            py_nvosd_circle_params.circle_color.set(0.0, 0.0, 1.0, 1.0)
            py_nvosd_circle_params.has_bg_color = 1
            py_nvosd_circle_params.bg_color.set(0.0, 0.0, 0.0, 1.0)
            py_nvosd_circle_params.xc = int(landmarks[j * 2]) if int(landmarks[j * 2]) > 0 else 0
            py_nvosd_circle_params.yc = int(landmarks[j * 2 + 1]) if int(landmarks[j * 2 + 1]) > 0 else 0
            py_nvosd_circle_params.radius=2
            ccount = ccount + 1
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)

    display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)       
    display_meta.num_labels = 1
    py_nvosd_text_params = display_meta.text_params[0]
    # Setting display text to be shown on screen
    # Note that the pyds module allocates a buffer for the string, and the
    # memory will not be claimed by the garbage collector.
    # Reading the display_text field here will return the C address of the
    # allocated string. Use pyds.get_string() to get the string content.
    py_nvosd_text_params.display_text = "Frame Number={} Number of Objects={}".format(frame_number, num_rects)

    # Now set the offsets where the string should appear
    py_nvosd_text_params.x_offset = 10
    py_nvosd_text_params.y_offset = 12

    # Font , font-color and font-size
    py_nvosd_text_params.font_params.font_name = "Serif"
    py_nvosd_text_params.font_params.font_size = 10
    # set(red, green, blue, alpha); set to White
    py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)

    # Text background color
    py_nvosd_text_params.set_bg_clr = 1
    # set(red, green, blue, alpha); set to Black
    py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
    # Using pyds.get_string() to get display_text as string
    # print(pyds.get_string(py_nvosd_text_params.display_text))
    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    try:
        l_frame=l_frame.next
    except StopIteration:
        break
		
return Gst.PadProbeReturn.OK

def main(args):
# Check input arguments
if len(args) != 2:
sys.stderr.write(“usage: %s \n” % args[0])
sys.exit(1)

global n_height, n_width, scale
n_height, n_width = 480, 640
scale = coor_scale(n_height, n_width, 1080, 1920)

# Standard GStreamer initialization
Gst.init(None)

# Create gstreamer elements
# Create Pipeline element that will form a connection of other elements
print("Creating Pipeline \n ")
pipeline = Gst.Pipeline()

if not pipeline:
    sys.stderr.write(" Unable to create Pipeline \n")

# Source element for reading from the file
print("Creating Source \n ")

source = Gst.ElementFactory.make("v4l2src", "usb-cam-source")
if not source:
    sys.stderr.write(" Unable to create Source \n")

caps_v4l2src = Gst.ElementFactory.make("capsfilter", "v4l2src_caps")
if not caps_v4l2src:
    sys.stderr.write(" Unable to create v4l2src capsfilter \n")

print("Creating Video Converter \n")

# videoconvert to make sure a superset of raw formats are supported
vidconvsrc = Gst.ElementFactory.make("videoconvert", "convertor_src1")
if not vidconvsrc:
    sys.stderr.write(" Unable to create videoconvert \n")

# nvvideoconvert to convert incoming raw buffers to NVMM Mem (NvBufSurface API)
nvvidconvsrc = Gst.ElementFactory.make("nvvideoconvert", "convertor_src2")
if not nvvidconvsrc:
    sys.stderr.write(" Unable to create Nvvideoconvert \n")

caps_vidconvsrc = Gst.ElementFactory.make("capsfilter", "nvmm_caps")
if not caps_vidconvsrc:
    sys.stderr.write(" Unable to create capsfilter \n")

# Create nvstreammux instance to form batches from one or more sources.
streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
if not streammux:
    sys.stderr.write(" Unable to create NvStreamMux \n")

# Use nvinfer to run inferencing on camera's output,
# behaviour of inferencing is set through config file
pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
if not pgie:
    sys.stderr.write(" Unable to create pgie \n")

# Use convertor to convert from NV12 to RGBA as required by nvosd
nvvidconv = Gst.ElementFactory.make("nvvideoconvert", "convertor")
if not nvvidconv:
    sys.stderr.write(" Unable to create nvvidconv \n")

# Create OSD to draw on the converted RGBA buffer
nvosd = Gst.ElementFactory.make("nvdsosd", "onscreendisplay")

if not nvosd:
    sys.stderr.write(" Unable to create nvosd \n")

# Finally render the osd output
if is_aarch64():
    print("Creating nv3dsink \n")
    sink = Gst.ElementFactory.make("nv3dsink", "nv3d-sink")
    if not sink:
        sys.stderr.write(" Unable to create nv3dsink \n")
else:
    print("Creating EGLSink \n")
    sink = Gst.ElementFactory.make("nveglglessink", "nvvideo-renderer")
    if not sink:
        sys.stderr.write(" Unable to create egl sink \n")

# queue1=Gst.ElementFactory.make("queue","queue1")
# queue2=Gst.ElementFactory.make("queue","queue2")
# queue3=Gst.ElementFactory.make("queue","queue3")
# queue4=Gst.ElementFactory.make("queue","queue4")
# queue5=Gst.ElementFactory.make("queue","queue5")
# queue6=Gst.ElementFactory.make("queue","queue6")
# queue7=Gst.ElementFactory.make("queue","queue7")
# pipeline.add(queue1)
# pipeline.add(queue2)
# pipeline.add(queue3)
# pipeline.add(queue4)
# pipeline.add(queue5)
# pipeline.add(queue6)
# pipeline.add(queue7)


print("Playing cam %s " %args[1])
caps_v4l2src.set_property('caps', Gst.Caps.from_string("video/x-raw, framerate=30/1"))
caps_vidconvsrc.set_property('caps', Gst.Caps.from_string("video/x-raw(memory:NVMM)"))
source.set_property('device', args[1])
streammux.set_property('width', 1920)
streammux.set_property('height', 1080)
streammux.set_property('batch-size', 1)
streammux.set_property('batched-push-timeout', 4000000)
pgie.set_property('config-file-path', "config_ds_retinaface.txt")

pgie_batch_size=pgie.get_property("batch-size")

if(pgie_batch_size != 1):
    print("WARNING: Overriding infer-config batch-size",pgie_batch_size," with number of sources ", 1," \n")
    pgie.set_property("batch-size", pgie_batch_size)
# Set sync = false to avoid late frame drops at the display-sink
sink.set_property('sync', False)

print("Adding elements to Pipeline \n")
pipeline.add(source)
pipeline.add(caps_v4l2src)
pipeline.add(vidconvsrc)
pipeline.add(nvvidconvsrc)
pipeline.add(caps_vidconvsrc)
pipeline.add(streammux)
pipeline.add(pgie)
pipeline.add(nvvidconv)
pipeline.add(nvosd)
pipeline.add(sink)

# we link the elements together
# v4l2src -> nvvideoconvert -> mux -> 
# nvinfer -> nvvideoconvert -> nvosd -> video-renderer
print("Linking elements in the Pipeline \n")
source.link(caps_v4l2src)
caps_v4l2src.link(vidconvsrc)
vidconvsrc.link(nvvidconvsrc)
nvvidconvsrc.link(caps_vidconvsrc)

sinkpad = streammux.get_request_pad("sink_0")
if not sinkpad:
    sys.stderr.write(" Unable to get the sink pad of streammux \n")
srcpad = caps_vidconvsrc.get_static_pad("src")
if not srcpad:
    sys.stderr.write(" Unable to get source pad of caps_vidconvsrc \n")
srcpad.link(sinkpad)
streammux.link(pgie)
pgie.link(nvvidconv)
nvvidconv.link(nvosd)
nvosd.link(sink)

# create an event loop and feed gstreamer bus mesages to it
loop = GLib.MainLoop()
bus = pipeline.get_bus()
bus.add_signal_watch()
bus.connect ("message", bus_call, loop)

# Lets add probe to get informed of the meta data generated, we add probe to
# the sink pad of the osd element, since by that time, the buffer would have
# had got all the metadata.
osdsinkpad = nvosd.get_static_pad("sink")
if not osdsinkpad:
    sys.stderr.write(" Unable to get sink pad of nvosd \n")

osdsinkpad.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe, 0)

# start play back and listen to events
print("Starting pipeline \n")
pipeline.set_state(Gst.State.PLAYING)
try:
    loop.run()
except:
    pass
# cleanup
pipeline.set_state(Gst.State.NULL)

if name == ‘main’:
sys.exit(main(sys.argv))

Could you share the DeepStream config files ?

what the model type you use ? tf or onnx

Is there a corresponding file missing in the config file

how the retina_r50.engine is generated ?

thx

config deepstream :
[property]

gpu-id=0
#0=RGB, 1=BGR
model-color-format=1
model-engine-file=./retinaface/retina_r50.engine
labelfile-path=./retinaface/labels.txt

process-mode=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=1
gie-unique-id=1
network-type=0
output-blob-names=prob

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

#cluster-mode=2
maintain-aspect-ratio=1
batch-size=1
num-detected-classes=1
output-tensor-meta=1

custom detection parser

parse-bbox-func-name=NvDsInferParseCustomRetinaface
custom-lib-path=./retinaface/nvdsinfer_customparser/libnvdsinfer_custom_impl_retinaface.so
net-scale-factor=1.0
offsets=104.0;117.0;123.0
force-implicit-batch-dim=0

number of consecutive batches to skip for inference

interval=0

[class-attrs-all]

bbox threshold

pre-cluster-threshold=0.6

nms threshold

post-cluster-threshold=0.4

nms-iou-threshold=0.5

convert .engine repo: tensorrtx/retinaface at master · wang-xinyu/tensorrtx · GitHub

1.Please make sure that the build engine uses the same device as running deepstream.
Because after the model is converted, the same operator is different on different platforms.

2.As your description,maybe you can export model of retinaface from pytorch to onnx,deepstream support build engine from onnx.

3.Fine-tune a pretrained retinanet model with your dataset is also supported

There is no update from you for a period,
assuming this is not an issue anymore.
Hence we are closing this topic.
If need further support, please open a new one. Thanks