DeepStream multiple tiles not working

On the Tesla platform, I ran the Deepstream demo app (deepstream-app) with a couple of configurations:

a) A single stream, displayed on a single tile, and b) 4 streams, displayed on 4 tiles.

In both cases, the pipeline of tasks is the same: Primary inference detects vehicles, and the secondary inference detects the make, color, type of vehicle.
This works well in the single stream case. However, in the 4 stream case, the primary inference draws boxes, but the secondary inference doesn’t show any labels at all.

I suspected that this might be due to the objects taking up a smaller number of pixels, and therefore reduced the minw and minh parameters for the secondary inference engines. Even then, the labels are not seen. Any ideas?

How about the config “source4_720p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt” ?

Yes, that’s what I used for case b) above.

Thanks.

Hi,
Can you paste the details of your config source4_720p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt?

Prashanth
Amy did not reproduce your issue. You can find the difference as the above question.

I’m sending the output of the run to a file, rather than on the local display, because I’m running on the cloud. Perhaps this makes a difference?

As requested by Amy, here’s the config file I used:

[application]
enable-perf-measurement=0
perf-measurement-interval-sec=5
flow-original-resolution=1
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=2
columns=2
width=1280
height=720
gpu-id=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://…/…/streams/sample_720p.mp4
num-sources=4
gpu-id=0

[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0

[sink1]
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4

only SW mpeg4 is supported right now.

codec=1
sync=0
bitrate=2000000
output-file=out2.mp4
source-id=0

[osd]
enable=1
gpu-id=0
osd-mode=1
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0

[primary-gie]
enable=1
gpu-id=0
net-scale-factor=0.0039215697906911373
model-file=…/…/models/Primary_Detector/resnet10.caffemodel
proto-file=…/…/models/Primary_Detector/resnet10.prototxt
model-cache=…/…/models/Primary_Detector/resnet10.caffemodel_b4_int8.cache
labelfile-path=…/…/models/Primary_Detector/labels.txt
int8-calib-file=…/…/models/Primary_Detector/cal_trt4.bin
net-stride=16
batch-size=4

0=FP32, 1=INT8, 2=FP16 mode

network-mode=1
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
num-classes=4
class-thresholds=0.2;0.2;0.1;0.2
class-eps=0.2;0.2;0.2;0.2
class-group-thresholds=1;1;1;1
roi-top-offset=0;0;0;0
roi-bottom-offset=0;0;0;0
detected-min-w=0;0;0;0
detected-min-h=0;0;0;0
detected-max-w=1280;1280;1280;1280
detected-max-h=720;720;720;720
interval=0
gie-unique-id=1
parse-func=4
output-bbox-name=conv2d_bbox
output-blob-names=conv2d_cov
parser-bbox-norm=35.0;35.0
#config-file=config_infer_resnet.txt

[tracker]
enable=1
tracker-width=640
tracker-height=368
gpu-id=0

[secondary-gie0]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel
proto-file=…/…/models/Secondary_VehicleTypes/resnet18.prototxt
model-cache=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_int8.cache
mean-file=…/…/models/Secondary_VehicleTypes/mean.ppm
labelfile-path=…/…/models/Secondary_VehicleTypes/labels.txt
int8-calib-file=…/…/models/Secondary_VehicleTypes/cal_trt4.bin
gpu-id=0
batch-size=16
num-classes=6
network-mode=1
detected-min-w=32
detected-min-h=32
detected-max-w=1280
detected-max-h=720
model-color-format=1
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[secondary-gie1]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_CarColor/resnet18.caffemodel
proto-file=…/…/models/Secondary_CarColor/resnet18.prototxt
model-cache=…/…/models/Secondary_CarColor/resnet18.caffemodel_b16_int8.cache
mean-file=…/…/models/Secondary_CarColor/mean.ppm
labelfile-path=…/…/models/Secondary_CarColor/labels.txt
int8-calib-file=…/…/models/Secondary_VehicleTypes/cal_trt4.bin
batch-size=16
network-mode=1
detected-min-w=32
detected-min-h=32
detected-max-w=1280
detected-max-h=720
model-color-format=1
num-classes=12
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[secondary-gie2]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_CarMake/resnet18.caffemodel
proto-file=…/…/models/Secondary_CarMake/resnet18.prototxt
model-cache=…/…/models/Secondary_CarMake/resnet18.caffemodel_b16_int8.cache
mean-file=…/…/models/Secondary_CarMake/mean.ppm
labelfile-path=…/…/models/Secondary_CarMake/labels.txt
int8-calib-file=…/…/models/Secondary_CarMake/cal_trt4.bin
batch-size=16
network-mode=1
num-classes=24
detected-min-w=32
detected-min-h=32
detected-max-w=1280
detected-max-h=720
model-color-format=1
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[tests]
file-loop=0

Hi,
for dump to file, in single instance mode you will see the sgie’s output
you will not see sgie output only in case of batched mode
sgie is attaching outputs but they are just not getting displayed through the OSD component, and this is deliberately done for increasing performance

Thanks

You can change the deepstream-app source code to display sige label text when you can get deepstream 3.0 release.

Set appCtx[0]->show_bbox_text = TRUE; in deepstream_app_main.c before g_main_loop_new

Thank you. When is DeepStream 3.0 expected? Is it only for Tesla, or for Jetson also?

Only for tesla.
Next step, the version for Jetson will come out.