Deepstream 6.3 : Too long to start with sample-app (resent_tracker)

[kim_de]

Please provide the following info (tick the boxes after creating this topic):
Software Version
[O] DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[O] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
[O] DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
[O] 1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[O] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Question.
Hello, I’m running the command ‘deepstream-app -c ./source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8’, which is running the camera data and detecting the object.
But the problem is it takes time too long like 5 minutes.
Can you tell me how to reduce the time using that command??
Thanks.

[fanzh]

could you share the whole log? at the first run, the application will cost some tiem to generate NVIDIA TRT engine for every model. after the first run, the application will load the engines directly instead of regenerating.

[kim_de]

Can you tell me how to log?
You mentioned that only the first run will takes some time for generating TRT engine,
but the issue I’m facing is that it seems to create a TRT engine every time when I run it.
So every time it takes 5min.

[fanzh]

1. deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.yml >1.log 2>1.log
2. you can correct model-engine-file with the correct the engine path.

[kim_de]

I just attached my ‘source4 .txt file’.
Can you give me any advice about how to modify the ‘model-engine-path’ to reduce the time to start the engines?
source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt (5.8 KB)

[fanzh]

1. after the first run, TensorRT engines will be generated, then you can set the engine path to model-engine-path. take “model-engine-file=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine” for example, you can enter to /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app and check if the path is valid by “ll …/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine”.
   2.if still can’t work, please provide a whole log mentioned in my comment.

[kim_de]

I just attached the 1.log file.
Please review.

Thanks
1.log (529.8 KB)

[fanzh]

WARNING: Serialize engine failed because of file path: /opt/nvidia/deepstream/deepstream-6.3/samples/models/Primary_Detector/resnet10.caffemodel_b4_gpu0_int8.engine opened error

As the log in 1.log shown, the app failed to save engine. please make sure the app has permission to save the engine in that directory. the please check if the engine is generated.

[kim_de]

1. I checked all the model engines listed in the file source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt, but they already exist.

2. I just logged the data again.
   1.log (530.4 KB)

I am unable to change the permissions for the directory (/opt/nvidia/deepstream/deepstream-6.3/samples/configs/deepstream-app). Therefore, I logged the file in the home directory and used the following command:

cmd : deepstream-app -c /opt/nvidia/deepstream/deepstream-6.3/samples/configs/deepstream-app/source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt > 1.log 2 > 1.log

[fanzh]

please refer to my last comment. the app can’t open the file to Serialize the engine. please make sure the app has permission to save the engine to /opt/nvidia/deepstream/deepstream-6.3/samples/models/Primary_Detector.

[kim_de]

Could you provide guidance on the command to grant permission for the app to save the engine?

[fanzh]

you can try Linux chown or chmod command-line.

[kim_de]

I just attached the ‘1.log’ file.
1.log (530.3 KB)

[fanzh]

please refer to this topic.

[kim_de]

I attached the new log file.

1.log (2.8 MB)

[fanzh]

is it the first run log? if so, it is expected because there is no engine file at the first time. if it is the first time, the app still failed to load resnet10.caffemodel_b2_gpu0_int8.engine because of “open error”. after the first run, is there the generated engine? could you share the result of “ll /opt/nvidia/deepstream/deepstream-6.3/samples/configs/deepstream-app/…/…/models/Primary_Detector/resnet10.caffemodel_b2_gpu0_int8.engine”?

[kim_de]

Sorry I miss the engine.
And I upload the new version of 1.log file.

file :
1.log (498.2 KB)

[kim_de]

I measured the execution time of the deepstream-app, which took 53 seconds to display the output. (Below image)

Is there a way to reduce this time to within 5 seconds?
I’ve already confirmed that the engine has been generated.

*The below image shows the execution process, which takes 53seconds.

[fanzh]

@kim_de

I’ve already confirmed that the engine has been generated.

from the log "failed to serialize cuda engine to file… " in the screenshot, the app failed to save engine. can you confirm the engine has been generated? the path is /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet10.caffemodel_b2_gpu0_int8.engine

[kim_de]

Here it is, and it has ‘resnet10.caffemodel_b2_gpu0_int8.engine’ file.