when I change my precision from init8 to fp16 || 32, tracker id is stable but bounding box jitters a lot, how do we fix it?
Hi,
Could you share us a video for the issue you mentioned?
So we can check it with our internal team for some suggestion.
Thanks.
Hi AastaLLL,
Thanks
Hi,
We want to reproduce this issue in our environment.
Would you mind to share your platform and configuration with us?
Thanks.
Hi AastaLLL,
I am using
AGX, 24 * 1920 *1080(H.264/High Profile) @ 60 FPS, bitrate 47,195kbps. 1 detector and 3 classifiers
I have offloaded 3 classifiers to DLA 0 and 1
I have tried with H265 * 30FPS and I had a similar issue.
I am using KLT now
It would be best if you post your entire deepstream-app config file so we can reproduce. Are you inferencing at every frame?
@ i have shared privately[dm]
NvDcf does multi camera re-identification ?
If you still sees the issue when using KLT, pls try increasing tracker-width and tracker-height.
NvDCF and other trackers in DeepStream are all single-camera tracker.
KLT is stable but tracker id changes often this will be an issue in vehicle counting, how do we fix the jitter issue with nvdcf
Hi,
Could you modify the configure file based on our source30_1080p_dec_infer-resnet_tiled_display_int8.txt
We want to reproduce this issue in our environment first.
Thanks.
I am using modified source30_1080p_dec_infer-resnet_tiled_display_int8.txt, I have changed from int 8 to FP16
source30_1080p_dec_infer-resnet_tiled_display_int8.txt default config with NVDCF works fine, but its single-source video and a single detector
@AastaLLL we replicated the issue on 8 * AGX kits
Hello,
Thanks for your update.
We are investigating this issue internally.
Will update more information once we got any news.
Thanks.
Hi,
We cannot reproduce this issue in our side.
Here are our configure for your reference:
# Copyright (c) 2018 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=2
uri=file://../../streams/sample_1080p_h264.mp4
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0
nvbuf-memory-type=0
[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector/resnet10.caffemodel_b30_int8.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
#Required by the app for SGIE, when used along with config-file property
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt
[tracker]
enable=1
tracker-width=480
tracker-height=272
#ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_iou.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_nvdcf.so
#ll-config-file required for IOU only
#ll-config-file=iou_config.txt
gpu-id=0
[tests]
file-loop=0
# Copyright (c) 2018 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
# Following properties are mandatory when engine files are not specified:
# int8-calib-file(Only in INT8)
# Caffemodel mandatory properties: model-file, proto-file, output-blob-names
# UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
# ONNX: onnx-file
#
# Mandatory properties for detectors:
# num-detected-classes
#
# Optional properties for detectors:
# enable-dbscan(Default=false), interval(Primary mode only, Default=0)
# custom-lib-path,
# parse-bbox-func-name
#
# Mandatory properties for classifiers:
# classifier-threshold, is-classifier
#
# Optional properties for classifiers:
# classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
# operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
# input-object-min-width, input-object-min-height, input-object-max-width,
# input-object-max-height
#
# Following properties are always recommended:
# batch-size(Default=1)
#
# Other optional properties:
# net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
# model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary),
# custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-file=../../models/Primary_Detector/resnet10.caffemodel
proto-file=../../models/Primary_Detector/resnet10.prototxt
model-engine-file=../../models/Primary_Detector/resnet10.caffemodel_b30_int8.engine
labelfile-path=../../models/Primary_Detector/labels.txt
int8-calib-file=../../models/Primary_Detector/cal_trt.bin
batch-size=30
process-mode=1
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
#parse-bbox-func-name=NvDsInferParseCustomResnet
#custom-lib-path=/path/to/libnvdsparsebbox.so
#enable-dbscan=1
[class-attrs-all]
threshold=0.2
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.2
#minBoxes=3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0
## Per class configuration
#[class-attrs-2]
#threshold=0.6
#eps=0.5
#group-threshold=3
#roi-top-offset=20
#roi-bottom-offset=10
#detected-min-w=40
#detected-min-h=40
#detected-max-w=400
#detected-max-h=800
Is there any missing in our test?
Thanks.
Hi Aasta,
Default demo runs fine with a single video
source30_1080p_dec_infer-resnet_tiled_display_int8.txt with singe source(video) works fine.
*source30_1080p_dec_infer-resnet_tiled_display_int8.txt has only 1 detector
Problem is only when we add multiple sources with different videos
I can give you VPN access to our lab
Can you try with
- Multiple video sources with different videos.
- Add secondary classifiers.
Primary classifier
[primary-gie]
enable=1
gpu-id=0
model-engine-file=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b16_fp16.engine
batch-size=16
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt
Secondary classifiers
[secondary-gie0]
enable=1
model-engine-file=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_int8.engine
labelfile-path=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_VehicleTypes/labels.txt
gpu-id=0
batch-size=16
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehicletypes.txt
[secondary-gie1]
enable=1
model-engine-file=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_CarColor/resnet18.caffemodel_b16_int8.engine
labelfile-path=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_CarColor/labels.txt
batch-size=16
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carcolor.txt
[secondary-gie2]
enable=1
model-engine-file=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_CarMake/resnet18.caffemodel_b16_int8.engine
labelfile-path=/opt/nvidia/deepstream/deepstream-4.0/samples/models/Secondary_CarMake/labels.txt
batch-size=16
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carmake.txt
Hello Ravi,
I noticed that the PGIE interval is set to 4: interval=4 Can you reduce this to 2 or 1?
We found a bug in the release version of NvDCF, and it may cause such jitter in case of interval>0. We fixed it and the next release version will be much smoother.
Can you let me know what does interval mean in this context, does it mean the object detector running for 4 frames or of that sort?
Could you also specify where the documentation for config files are present?
Hi Pshin,
interval=0 is stable.
https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/index.html#page/DeepStream_Development_Guide%2Fdeepstream_quick_start.html%23wwpID0E0GB0HA
Hey @pshin, can we get any info on the timeline for next release?
We have a documentation at [DeepStream Development Guide]
Please refer to interval at PGIE section.
I cannot disclose any future plans, sorry. Please refer to a public announcement related to DeepStream. But, I guess we are trying to make the releases in a regular basis.
Hello @pshin, 6 months later I still experience the jitter issue described above with the nvdcf tracker that came with DeepStream 5.0DP if I set the interval parameter of the tracker to anything greater than zero. Do you have some update on the fix? Maybe we can patch the tracker somehow?
I created a small log that shows how many objects are tracked reported on the metadata stream for each from, running on the sample_720p.h264 example stream. I set interval=3 in the tracker config so the tracker should run only on each 4th frame. It is clearly visible that the number of objects reported is fluctuating: every 4th frame (I guess those ones when the tracker run) reports significantly more objects.
Frame Number = 496, FPS = 11.13, objects = 15
Frame Number = 497, FPS = 11.10, objects = 8
Frame Number = 498, FPS = 11.12, objects = 8
Frame Number = 499, FPS = 11.13, objects = 9
Frame Number = 500, FPS = 11.13, objects = 15
Frame Number = 501, FPS = 11.10, objects = 7
Frame Number = 502, FPS = 11.12, objects = 8
Frame Number = 503, FPS = 11.14, objects = 6
Frame Number = 504, FPS = 11.13, objects = 14
Frame Number = 505, FPS = 11.10, objects = 6
Frame Number = 506, FPS = 11.12, objects = 8
Frame Number = 507, FPS = 11.13, objects = 8
Frame Number = 508, FPS = 11.13, objects = 14
Frame Number = 509, FPS = 11.10, objects = 6
Frame Number = 510, FPS = 11.12, objects = 8
Frame Number = 511, FPS = 11.13, objects = 8
Frame Number = 512, FPS = 11.13, objects = 13
Frame Number = 513, FPS = 11.10, objects = 7
Frame Number = 514, FPS = 11.12, objects = 6
Frame Number = 515, FPS = 11.13, objects = 7
Frame Number = 516, FPS = 11.13, objects = 12
Frame Number = 517, FPS = 11.10, objects = 5
Frame Number = 518, FPS = 11.12, objects = 5
Frame Number = 519, FPS = 11.13, objects = 6
1 Like