Retail Object Detection - Issues Testing

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Hi,

I hope you are well. I am running a SeedStudio reComputer J3011 (Jetson Orin Nano 8GB) with Jetpack 6.2 Flashed. I have Deepstream 7.1 Installed and I am trying to test the retail_detector_100.onnx model, through my RTSP stream.

I have converted it to a TRT file but get a bunch of errors when trying to run deepstream. I am fairly new to this, would you be able to give any advice on this and also check my config files?

I have config_infer_primary.txt:

[property]
gpu-id=0
net-scale-factor=1
offsets=0;0;0
model-color-format=0
model-engine-file=/home/pj/retail-ai/models/retail_model.trt
labelfile-path=/home/pj/retail-ai/recognitionv2_name_list.txt
batch-size=1
gie-unique-id=1
network-input-order=1
infer-dims=3;416;416
maintain-aspect-ratio=1
network-mode=2
num-detected-classes=100
network-type=1
interval=0
network-input-order=1
output-blob-names=num_detections;detection_boxes;detection_scores;detection_classes
custom-lib-path=/home/pj/retail-ai/repos/deepstream_tao_apps/post_processor/libnvds_infercustomparser_tao.so

[class-attrs-all]
pre-cluster-threshold=0.1

Then I am trying to run from source1_usb_dec_infer_resnet_int8.txt:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=640
height=480

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
camera-width=640
camera-height=480
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=0
uri=rtsp://admin:password@10.0.133.219:554/cam/realmonitor?channel=6&subtype=0

[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink/nv3dsink(Jetson only) 3=File 4=RTSPStreaming 5=nvdrmvideosink
type=2
sync=0
conn-id=0
width=0
height=0
plane-id=1
source-id=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=2000000
sw-preset
profile=0
output-file=out.mp4
source-id=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=nvdrmvideosink
type=4
#1=h264 2=h265
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
rtsp-port=8554
udp-port=5400

[osd]
enable=1
border-width=2
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0

[streammux]
live-source=1
batch-size=1
batched-push-timeout=40000
width=960
height=540

[primary-gie]
enable=1
model-engine-file=/home/pj/retail-ai/models/retail_model.trt
batch-size=1
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
config-file=/home/pj/retail-ai/configs/config_infer_primary.txt
labelfile-path=/home/pj/retail-ai/recognitionv2_name_list.txt

[tests]
file-loop=0

I am getting the following errors:

** INFO: <create_udpsink_bin:646>: Could not create HW encoder. Falling back to SW encoder

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

WARN !! Invalid mode selected, Falling back to GPU

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-7.1/lib/libnvds_nvmultiobjecttracker.so

[NvTrackerParams::getConfigRoot()] !!![WARNING] Can’t open config file (/home/pj/retail-ai/configs/config_tracker.txt). Will go ahead with default values

[NvMultiObjectTracker] Initialized

Setting min object dimensions as 16x16 instead of 1x1 to support VIC compute mode.

WARNING: [TRT]: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.

0:00:01.101515930 25600 0xaaaabc35f6f0 INFO nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2092> [UID = 1]: deserialized trt engine from :/home/pj/retail-ai/models/retail_model.trt

Implicit layer support has been deprecated

INFO: [Implicit Engine Info]: layers num: 0

WARNING: Backend context bufferIdx(0) request dims:1x416x416x3 is out of range, [min: 1x3x224x224, max: 1x3x224x224]

0:00:01.101668572 25600 0xaaaabc35f6f0 WARN nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:2050> [UID = 1]: backend can not support dims:416x416x3

0:00:01.101691420 25600 0xaaaabc35f6f0 WARN nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2201> [UID = 1]: deserialized backend context :/home/pj/retail-ai/models/retail_model.trt failed to match config params, trying rebuild

0:00:01.197310654 25600 0xaaaabc35f6f0 INFO nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2106> [UID = 1]: Trying to create engine from model files

ERROR: failed to build network since there is no model file matched.

ERROR: failed to build network.

0:00:09.617219218 25600 0xaaaabc35f6f0 ERROR nvinfer gstnvinfer.cpp:678:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2126> [UID = 1]: build engine file failed

0:00:10.001817767 25600 0xaaaabc35f6f0 ERROR nvinfer gstnvinfer.cpp:678:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2213> [UID = 1]: build backend context failed

0:00:10.001895880 25600 0xaaaabc35f6f0 ERROR nvinfer gstnvinfer.cpp:678:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1351> [UID = 1]: generate backend failed, check config file settings

0:00:10.002735603 25600 0xaaaabc35f6f0 WARN nvinfer gstnvinfer.cpp:914:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance

0:00:10.002765107 25600 0xaaaabc35f6f0 WARN nvinfer gstnvinfer.cpp:914:gst_nvinfer_start:<primary_gie> error: Config file path: /home/pj/retail-ai/configs/config_infer_primary.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

** ERROR: main:713: Failed to set pipeline to PAUSED

Quitting

ERROR from primary_gie: Failed to create NvDsInferContext instance

Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(914): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:

Config file path: /home/pj/retail-ai/configs/config_infer_primary.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

[NvMultiObjectTracker] De-initialized
App run failed

Any help or advice would be appreciated!

Thanks.

3

The model input dim is different the input dim in your configure file. You need set the right value in config_infer_primary.txt based on your model.

4

Thanks!

I managed to get the model running but as you can see:

image
image1440×1079 211 KB

It seems there is a big bounding box around my entire RTSP stream. This model was built with 416x416 MAX dims. Do you think my camera is too far away?

How can I train this model on my own products, and would it be sufficient from this distance?

Regards,

PJ Pretorius.

5

What is the expect output of the model? Can you share more information about the model?

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks