Use re trainned model in deepstream

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello,

• Hardware : x86 dGPU
• Network Type : YoloV3

I used Jupyter YOLOv3’s TAO procedure to train a model with new images successfully.
I would like to integrate the export model into DeepStream.
I found an example configuration here: YOLOv3 — Transfer Learning Toolkit 3.0 documentation

I modified it but the network fails to be created when launching Deepstream.

My config file (i do not share my API KEY):

[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=/home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/labels.txt
model-engine-file=/home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/trt.engine
tlt-model-key=MY_API_KEY
infer-dims=3;384;1248
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
num-detected-classes=1
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
#no cluster
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=/home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/libnvds_infercustomparser.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

Error from console output :

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
[NvMultiObjectTracker] Initialized
ERROR: [TRT]: 1: [stdArchiveReader.cpp::StdArchiveReader::40] Error Code 1: Serialization (Serialization assertion stdVersionRead == serializationVersion failed.Version tag does not match. Note: Current Version: 213, Serialized Engine Version: 232)
ERROR: [TRT]: 4: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:1528 Deserialize engine failed from file: /home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/trt.engine
0:00:01.659447281 35607 0x7f34ecc92610 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/trt.engine failed
0:00:01.685869645 35607 0x7f34ecc92610 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/trt.engine failed, try rebuild
0:00:01.685889080 35607 0x7f34ecc92610 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:860 failed to build network since there is no model file matched.
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:799 failed to build network.
0:00:02.469404375 35607 0x7f34ecc92610 ERROR nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1943> [UID = 1]: build engine file failed
0:00:02.495544309 35607 0x7f34ecc92610 ERROR nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2029> [UID = 1]: build backend context failed
0:00:02.495565027 35607 0x7f34ecc92610 ERROR nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1266> [UID = 1]: generate backend failed, check config file settings
0:00:02.495848660 35607 0x7f34ecc92610 WARN nvinfer gstnvinfer.cpp:846:gst_nvinfer_start: error: Failed to create NvDsInferContext instance
0:00:02.495860122 35607 0x7f34ecc92610 WARN nvinfer gstnvinfer.cpp:846:gst_nvinfer_start: error: Config file path: /home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/config_infer_drone_trt.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
[NvMultiObjectTracker] De-initialized
Error: gst-resource-error-quark: Failed to create NvDsInferContext instance (1): gstnvinfer.cpp(846): gst_nvinfer_start (): /GstPipeline:pipeline0/GstNvInfer:primary-inference:
Config file path: /home/loic/Documents/Dev/DeepExaCam/ExavisionSources/ExaCamBlackBox/data/drone/config_infer_drone_trt.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

2

I think the problem comes from the versions of TensorFlow used by the container in jupyter (8.5.1-1) and that of DeepStream on my machine (8.4.1-1).
Is there a TAO image with TensofrFlow 8.4.1-1, I can’t find one ?

3

Above error implies that the TRT version does not match.
A simple solution is to download tao-converter to generate tensorrt engine under your inference environment and then run the tensorrt engine in the same environment.

4

I find a most simple solution, simply comment model-engine-file and add tlt-encoded-model property.
At startup deepstream create tensorRT engine. It’s work but when i run inference i have many bounding box around same object, is it normal ?

5

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Yes, it is also a simple way to let deepstream generate tensorrt engine for you.

Can you run with official deepstream app GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream ? In your above config, seems that something is different. For example, libnvds_infercustomparser.so is different from https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/blob/master/configs/yolov3_tao/pgie_yolov3_tao_config.txt#L49

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.