Description
Serialization of model engine from ETLT fails
Environment
TensorRT Version: 8.5.2.2
GPU Type: NVIDIA GeForce GTX 1650 Mobile
Nvidia Driver Version: 525.78.01
CUDA Version: 11.8
CUDNN Version: 8.7.0
Operating System + Version: Linux Ubuntu 20.04 LTS
Python Version (if applicable): 3.8
DeepStream Version: 6.2
This is the error I get every time I run my DeepStream pipeline. This used to work fine when I was using DeepStream 6.1 with TensorRT 8.2.5.1. What changes do I have to make for this to work with 6.2 as well?
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1487 Deserialize engine failed because file path: /home/forgottenlight/GitHub/analytics/configs/pgies/../../data/pgies/resnet34_peoplenet_int8.etlt_b4_gpu0_int8.engine open error
0:00:02.768937715 56665 0x7fb2d1c63430 WARN nvinfer gstnvinfer.cpp:677:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/home/forgottenlight/GitHub/analytics/configs/pgies/../../data/pgies/resnet34_peoplenet_int8.etlt_b4_gpu0_int8.engine failed
0:00:02.798638313 56665 0x7fb2d1c63430 WARN nvinfer gstnvinfer.cpp:677:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/home/forgottenlight/GitHub/analytics/configs/pgies/../../data/pgies/resnet34_peoplenet_int8.etlt_b4_gpu0_int8.engine failed, try rebuild
0:00:02.799148319 56665 0x7fb2d1c63430 INFO nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
ERROR: [TRT]: 3: [builder.cpp::~Builder::307] Error Code 3: API Usage Error (Parameter check failed at: optimizer/api/builder.cpp::~Builder::307, condition: mObjectCounter.use_count() == 1. Destroying a builder object before destroying objects it created leads to undefined behavior.
)
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
ERROR: [TRT]: UffParser: UFF buffer empty
parseModel: Failed to parse UFF model
ERROR: tlt/tlt_decode.cpp:358 Failed to build network, error in model parsing.
ERROR: [TRT]: 3: [builder.cpp::~Builder::307] Error Code 3: API Usage Error (Parameter check failed at: optimizer/api/builder.cpp::~Builder::307, condition: mObjectCounter.use_count() == 1. Destroying a builder object before destroying objects it created leads to undefined behavior.
)
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:728 Failed to create network using custom network creation function
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:794 Failed to get cuda engine from custom library API
0:00:04.939375357 56665 0x7fb2d1c63430 ERROR nvinfer gstnvinfer.cpp:674:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1943> [UID = 1]: build engine file failed
ERROR: [TRT]: 2: [logging.cpp::decRefCount::65] Error Code 2: Internal Error (Assertion mRefCount > 0 failed. )
corrupted size vs. prev_size while consolidating
Here’s my config file:
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
tlt-model-key=tlt_encode
tlt-encoded-model=../../data/pgies/resnet34_peoplenet_int8.etlt
labelfile-path=../../data/labels/labels_peoplenet.txt
model-engine-file=../../data/pgies/resnet34_peoplenet_int8.etlt_b4_gpu0_int8.engine
int8-calib-file=../../data/pgies/resnet34_peoplenet_int8.txt
infer-dims=3;544;960;
uff-input-blob-name=input_1
batch-size=4
process-mode=1
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=1
num-detected-classes=3
cluster-mode=2
interval=0
gie-unique-id=1
output-blob-names=output_bbox/BiasAdd;output_cov/Sigmoid
#Use the config params below for NMS clustering mode
[class-attrs-all]
topk=20
nms-iou-threshold=0.5
pre-cluster-threshold=0.2
## Per class configurations
[class-attrs-0]
topk=20
nms-iou-threshold=0.7
pre-cluster-threshold=0.2