Please provide complete information as applicable to your setup.
**• x86 with Nvidia L4
• DeepStream Version 7.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 10.8
• NVIDIA GPU Driver Version (valid for GPU only) 535.183.01
• Issue Type( questions, new requirements, bugs)
I am trying to run a deepstream pipeline consisting of a primary detector (custom RT-DETR model) and a tracker on a sequence of images.
This pipeline works well on an Orin with Deepstream 6.4 and JetPack 6.0.
The model engine has been rebuilt on the L4 machine and
/usr/src/tensorrt/bin/trtexec --loadEngine=models/rtdetr/model.engine
shows that the engine file, in principle, is working
configuration for the engine is as follows:
################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
# Following properties are mandatory when engine files are not specified:
# int8-calib-file(Only in INT8)
# Caffemodel mandatory properties: model-file, proto-file, output-blob-names
# UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
# ONNX: onnx-file
#
# Mandatory properties for detectors:
# num-detected-classes
#
# Optional properties for detectors:
# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0)
# custom-lib-path,
# parse-bbox-func-name
#
# Mandatory properties for classifiers:
# classifier-threshold, is-classifier
#
# Optional properties for classifiers:
# classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
# operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
# input-object-min-width, input-object-min-height, input-object-max-width,
# input-object-max-height
#
# Following properties are always recommended:
# batch-size(Default=1)
#
# Other optional properties:
# net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
# model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),
# custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.
[property]
gpu-id=0
net-scale-factor=0.00392156862745098
offsets=0.0;0.0;0.0
#tlt-model-key=tlt_encode
network-mode=0
model-engine-file=../../models/rtdetr/model.engine
#model-engine-file=../../models/rtdetr/model.plan
labelfile-path=../../models/rtdetr/labels.txt
force-implicit-batch-dim=0
batch-size=1
process-mode=1
model-color-format=0
#yolov8 specifics
maintain-aspect-ratio=0
symmetric-padding=1
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=../../models/rtdetr/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet
# everything except person (index 0 in labels.txt)
filter-out-class-ids=1;2;3;4;5;6;7;8;9;10;11;12;13;14;15;16;17;18;19;20;21;22;23;24;25;26;27;28;29;30;31;32;33;34;35;36;37;38;39;40;41;42;43;44;45;46;47;48;49;50;51;52;53;54;55;56;57;58;59;60;61;62;63;64;65;66;67;68;69;70;71;72;73;74;75;76;77;78;79
output-tensor-meta=0
network-type=0
gie-unique-id=1
#infer-dims=3;544;960
infer-dims=3;640;640
uff-input-order=0
uff-input-blob-name=input_1
#output-blob-names=output_cov/Sigmoid;output_bbox/BiasAdd
cluster-mode=2
#filter-out-class-ids=1
#enable-dla=1
#use-dla-core=0
[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300
this uses a custom library, that also has been succesfully compiled on the L4 machine , using the same working source code that was used on the Orin device -where it is know to work.
What is different on the L4 machine is the execution of the pipeline, which produces the following:
[2025-02-12 21:00:09,827] [deepstream-pose-estimation] [INFO] loop start
0:00:00.517522468 37272 0x614d6a64e4c0 INFO nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2092> [UID = 1]: deserialized trt engine from :/home/felix/code/deepstream-pose-estimation/models/rtdetr/model.engine
Implicit layer support has been deprecated
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:327 [Implicit Engine Info]: layers num: 0
0:00:00.517579835 37272 0x614d6a64e4c0 INFO nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2195> [UID = 1]: Use deserialized engine model: /home/felix/code/deepstream-pose-estimation/models/rtdetr/model.engine
0:00:00.532680777 37272 0x614d6a64e4c0 INFO nvinfer gstnvinfer_impl.cpp:343:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:.//config/python-pipeline/rtdetr_dynamic.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
[NvMultiObjectTracker] Loading TRT Engine for tracker ReID...
Using GPU 0 (NVIDIA L4, 58 SMs, 1536 th/SM max, CC 8.9, ECC on)
[NvMultiObjectTracker] Loading Complete!
[NvMultiObjectTracker] Initialized
Segmentation fault (core dumped)
Normally , for the Implicit Engine Info part I would expect the output of the input/output layers of the model here, but all I get is “layers num:0”
I also tried this with a vanilla fp32 version of peoplenet (with no custom output library) and run into the same issue.